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RANK 2 SHEAVES ON TORIC 3-FOLDS: CLASSICAL AND 

VIRTUAL COUNTS 

AMIN GHOLAMPOUR, MARTIJN KOOL, AND BENJAMIN YOUNG 


Abstract. Let A4 be the moduli space of rank 2 stable torsion free sheaves with Chern 
classes Ci on a smooth 3-fold X. When X is toric with torus T, we describe the T-fixed 
locus of the moduli space. Connected components of A4'^ with constant reflexive hulls 
are isomorphic to products of P^. We mainly consider such connected components, 
which typically arise for any ci, “low values” of C 2 , and arbitrary C 3 . 

In the classical part of the paper, we introduce a new type of combinatorics called 
double box configurations, which can be used to compute the generating function 7-{q) 
of topological Euler characteristics of Xi (summing over all C3). The combinatorics 
is solved using the double dimer model in a companion paper. This leads to explicit 
formulae for Z(g) involving the MacMahon function. 

In the virtual part of the paper, we define Donaldson-Thomas type invariants of 
toric Calabi-Yau 3-folds by virtual localization. The contribution to the invariant of 
an individual connected component of the T-fixed locus is in general not equal to its 
signed Euler characteristic due to T-fixed obstructions. Nevertheless, the generating 
function of all invariants is given by Z(g) up to signs. 
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1. Introduction 

Let X be an n-dimensional smooth projective variety over C with polarization H. Let 
M-xiPi Cl,..., Cn) denote the moduli space of rank r /i-stable torsion free sheaves on X 
with Chern classes Ci,..., See for the construction of the moduli space and its 
history. In this paper, we do not consider strictly semistable sheaves so this moduli space 
may be non-compact]^ Fixing ci,..., c„_i, one can consider the generating function 

(1) Zx,H,r,ci,...,c„_i(g) = ^ e(Alf(r,ci,...,c„))g"C 

Cn 

^A coherent sheaf J- is called torsion free if it has no non-zero subsheaves of dimension < dim A. 
^One way to ensure the absence of strictly semistable sheaves is by taking gcd(r, = 1. In 

this case, the moduli space is compact. 
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where e(-) denotes topological Euler characteristic. This is a Laurent series in q when n is 
even and q~^ when n is odd |GKll Prop. 3.6]. These generating functions are fascinating 
objects with links to combinatorics, number theory, and representation theory. We 
mention a few among many references. For n = 1, generating functions of Poincare 
polynomials were calculated by G. Harder and M. S. Narasimhan |HN] . U.V. Desale and 
S. Ramanan ing, and M. F. Atiyah and R. Bott |ABj . For n = 2 and r = 1, generating 
functions for Poincare polynomials were computed by L. Gottsche |Gotl] . For n = 2 
and r > 1 , these generating functions are related to (quasi-)modular forms and have 
been computed in numerous examples |Kly2 lYost IManll IMan2( IMan3( IMozj . This case 
is related to the S-duality conjecture of physics |VWj . For any n and r = 1, these 
generating functions lead to counting n-dimensional boxes as shown by J. Cheah [Che]. 


For n = 3 and X satisfying ^ 0, R. P. Thomas constructed a perfect 

obstruction theory on Mx('>"XiX2, Cs) |ThE|. When the moduli space is compact, this 
leads to a virtual cycle, which can be used to dehne deformation invariants. When X is 
in addition Calabi-Yau, the perfect obstruction theory is symmetric so the virtual cycle is 
0-dimensional. Its degree is known as a Donaldson-Thomas invariant. K. Behrend |Beh] 
showed the DT invariant is also equal to the Euler characteristic of D, C 2 , ca) 

weighted by a constructible function z/ 

deg([Wlf (r,Ci,C 2 ,C 3 )]'''") = e(Wlf (r, Ci, Cs, C 3 ), z/) = ^ A; e(iy~\{k})). 

kez 

Therefore, the generating function ([^ is related to DT theory. When X is a toric 
Calabi-Yau 3-fold and r = 1, D. Maulik, N. Nekrasov, A. Okounkov, and R. Pand- 
haripande [MNOPl] showed that the generating function of DT invariants (dehned by 
T-localization) is given by counting the T-£xed points of Ad^(l, Ci, C 2 , C 3 ) with signsj^ 
The signs can be seen as a consequence of weighing by the Behrend function |BFj . 

This paper is split up in two parts, which can be read more or less independently. The 
classical part is about ([^ in the case r = 2 and X is a toric 3-fold. The virtual part is 
concerned with rank 2 DT type invariants of a toric Calabi-Yau 3-fold and how these 
invariants relate to the Euler characteristics discussed in the classical part. 

Part I: Classical 


Little is known about the generating function ([^ for n > 2 and r > 1. We study 
the case n = 3, r = 2, and X is toric. In this case, X contains a dense open torus T 
acting on X and this action lifts to the moduli space. Elements of the fixed locus of 
the moduli space admit a T-equivariant structure (unique up to tensoring by a character 
of T). Equivariant sheaves on toric varieties were studied in depth by A. Klyachko 
|Klyl[ |Kly2| . He used his methods to calculate ([^ for X = and r = 2 obtaining the 
holomorphic part of a quasi-modular form of weight 3/2 |Kly2 , IVW] . Other localization 
calculations on (stacky/ordinary) toric surfaces appear in |Per2[ Oiol IGJKt IKoo2j . 

Let (■)* = J^om{-,Ox) denote the dual. Any torsion free sheaf X can be naturally 
embedded in its double dual Tl = X**, known as its reflexive hull. This leads to a short 
exact sequence 

—^ 0 , 


^A toric Calabi-Yau 3-fold cannot be projective. In this case the DT invariants are defined by the 
localization formula of m- 
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where Q is a sheaf of dimension < 1 . Reflexive hulls are reflexive sheaves^ A reflex¬ 
ive sheaf is torsion free, but much easier to describe than a general torsion free sheaf, 
because it is determined by its restriction to the complement of any closed subset of 
codimension > 2. Reflexive sheaves on a 3-fold are locally free outside a 0-dimensional 
subscheme. For general theory on reflexive sheaves, we refer to |Har2j . Klyachko showed 
that T-equivariant rank 2 reflexive sheaves have a straightforward description: they are 
described by attaching a flag of to each ray of the fan of X (his description works 
for any rank on any toric n-fold). Denote by ci, C 2 , cfl) the moduli space of rank 2 

/r-stable reflexive sheaves on X with Chern classes Ci, C 2 , C 3 . Taking double dual gives a 
map to the scheme theoretic disjoint union 

( 2 ) (■)** ; Aff ( 2 , Cl, C 2 , C 3 ) □ Af ( 2 , ci, c', c'). 


Reflexive hulls of members of a flat family do not need to form a flat family (already 
on surfaces), so this map is not a morphism. Nevertheless, by a result of J. Kollar 
IH, this map is constructible^ This is very useful for calculating Euler characteristics 
by cut-paste methods. On a surface, the cokernel Q is 0-dimensional and the double 
dual map leads to a product formula expressing ([^ as a “point contribution” times the 
generating function of reflexive (i.e. locally free) sheaves |Got2] . Two interesting things 
happen when n = 3 and r = 2 : the cokernel Q need not be 0-dimensional and TZ need 
not be locally free. Since we are interested in Euler characteristics, we only need to 
understand ([^ at the level of T-£xed points, where we can use Klyachko’s description 
of T-equivariant torsion free and reflexive sheaves. As already mentioned, T-equivariant 
reflexive sheaves are very well-behaved. Generating functions of A/'^(2, ci, C 2 , Cg) were 
studied in |GKlj . 

Equivariant torsion free sheaves are much harder to enumerate than equivariant re¬ 
flexive sheaves. At the level of closed points, the hbre of ([^ over a reflexive sheaf TZ is 
the Quot scheme 


Quotjf (A, C 2 , c'fl 


-2) "-a;) 


of cokernels 7Z ^ Q where Q has dimension < 1 and Chern classes C 2 , Cgj^ Suppose 7Z 
is T-equivariant, then T acts on Quotj^(A, C2, C3). The connected components of the T- 
hxed locus of Quot(A, C 2 , Cg) are indexed by a new type of combinatorial objects, which 
we call double box conhgurations (Dehnition 2.17). Moreover, each connected component 
is isomorphic to a product of P^’s (Proposition 2.11). When Q is 0-dimensional, i.e. C 2 = 
0, we solve the combinatorics completely in a companion paper |GKYj . 

One basic building block for our computations is Quotas (77, n): the Quot scheme of 
0-dimensional quotients 77 ^ Q of length n, where 77 is a T-equivariant rank 2 reflexive 
sheaf on C^. If 77 is locally free, it decompose^ as a sum of two line bundles 77 = Tg ©T 2 , 
which gives an additional C*-action by scaling the factors. Localization with respect to 
this action gives a well-known result 


(3) e(Quotc3 (77, 

n=0 


‘^A coherent sheaf T is called reflexive if the natural map T —>■ T** is an isomorphism. 

®More precisely, the domain can be written as a union of locally closed subschemes Ci such that on 
each Ci the map (•)** is a morphism. 

®The Chern classes q, c[ , 

^Due to Proposition 


2.4 


c" are related by equation (flO). 
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where M(q) denotes the MacMahon function counting 3D partitions. When TZ is singular, 
i.e. not locally free hence indecomposable^, this generating function is much harder to 
calculate. Then there are integers vi,V 2 ,V 3 > 0, which roughly speaking encode the 
weights of the three homogeneous generators of H^{TZ). In terms of these weights, we 
have: 


Theorem A. pTyl IGK^ 


^e(Quotc3(7^, n))q^ 

n=0 


V\ V2 V3 


i=l j=l k=l 


X — 

X — qi+j+k-2 ■ 


The triple product factor is the generating function of 3D partitions in a box of lengths 
Vi by V 2 by and is therefore a polynomial in q |Stal (7.109)]. The length of the 
singularity of 77 is V 1 V 2 V 3 (Proposition |2.4[ ). Therefore, in the singular case we get the 
“locally free answer” M{qY times a polynomial counting partitions “in the singularity 
of 77”. There are two proofs of this theorem: 

• Using C*Ulocahzation on Quotha (77, n) and the double dimer model. The hxed 
locus of Quotas (77, n) is described in this paper. The link to double dimers and 
how it leads to a proof of Theorem Inform the topic of the (purely combinatorial) 
companion paper pTYj . 

• Using Hall algebra techniques, a non-toric version of Theorem was recently 
proved by the hrst two authors in |GK2] . This proof provides a geometric ex¬ 
planation why counting inside the singularity of 77 and the MacMahon function 
occur. 


The combinatorial proof |GKY] was found before the geometric proof |GK2] . We briefly 


discuss these two proofs in Section 2.6 


What does this result tell us about generating function ([^ for n = 3 and r = 2? 
Recall that we £x Ci,C 2 and sum over all C 3 . If C 2 is chosen such that C 2 H is minimal 
with the property that there exists a rank 2 /i-stable torsion free sheaf on X with Chern 
classes Ci,C 2 , then Q is automatically 0-dimensional (Proposition 3.1). In this case, we 
get a structure theorem for ([^ (Proposition |3.2[ ). If in addition the polyhedron A(X) of 
the toric 3-fold X is small in the sense of Dehnition 3.3| (e.g. when rkPic(X) < 3), then 
all T-equivariant reflexive hulls in the moduli space are isolated T-fixed points and the 
formula simplihes (Theorem 3.5). A special case of this setting is X = and C 2 = 1. 
In principle, our approach also works for more general C 2 , but then the combinatorics 
becomes increasingly complex. We illustrate this for X = P^ and C 2 = 2. 


Theorem B. For X = P^ and Ci = —1, we have 
Z,,=i(g)=4(g-i + g)M(g-Y, 

2g-4 - g-2 + 1 - 4g2 + 3^^ + 5g® 


Zc2=2(7) - 12 I 


<M{q-^f 


(l-g 2)2 


Part II: Virtual 


Let y be a smooth toric Calabi-Yau 3-fold. Since Y is non-compact, we choose a toric 
compactihcation Y C X with H^{KV) 7 ^ 0 and a polarization H on X. Define the 
T-invariant open subset 


-^ycx(25 Cl, C2, C3) C Cl, C2, C3) 
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of torsion free sheaves on X such that T** jT = Q is supported on Y. Openness is 


proved in Proposition 4.3 Unlike the rank 1 case, this moduli space depends on the 
choice of X and H. This reflects dependence of stability on choice of polarization for 
higher rank. When C is a compact connected component of Xl^(2, ci, C 2 , ca)^, it has an 
induced virtual cycle by ins and one can consider the expression 


(4) 


'[C]- 


e(X^ 


where e(-) denotes T-equivariant Euler class and lY™ denotes the virtual normal bundle 
of C. These integrals depend on equivariant parameters Si, 52 , 53 . We are interested in 
connected components C, which also satisfy C C hAyizx- Unfortunately, the DT complex 
restricted to such a component C is not automatically symmetric. However, if we suppose 
in addition that all closed points of C have the same (i.e. isomorphic) reflexive hull 7^, 
then we can write Q as 

e(i?Hom(7?.,7?.)o 0 (Pc) X [ 

J[C]^ 


e(X™) e{R Hom(7^, 7^)o 0 (Pc) ’ 


where the second factor of this product is a number for 5 i + 52 + 53 = 0 (Proposition- 


Dehnition 4.11). We denote this number by DT((7). The reason for this is that after 


removing —i?Hom(7^, 7?.)o from the DT complex on C in the X-group, it becomes sym¬ 
metric (Proposition 4.6). We think of DT(C) as the contribution of C to rank 2 DT type 
invariants of yfl We have a precise conjecture for generating functions of DT(C) in any 
Chern classes on any toric Calabi-Yau 3-fold (Main Conjecture 6.1). 

For y = and hxed T-equivariant rank 2 reflexive hull TZ on consider all con¬ 
nected components C for which all its elements have constant reflexive hull TZ and 0- 
dimensional cokernel. Then the generating function of all DT(C) is the degree 0 rank 2 
equivariant vertex, which we denote by Wt^ 0^0 0 (g). We conjecture an explicit formula 
for it (Conjecture |6.9[): 


Conjecture C. 


W 


71 , 0 , 0,0 


( 7 ) 


51 + 52 “!“- 53—0 


M{qf 

M{qf nrii n;ii nr=i 


Ql+j+fc — 2 


if TZ is locally free 
if IZ is singular. 


This can be viewed as the degree 0 part of rank 2 DT theory on smooth toric 3-folds. 
This formula is in accordance with the expectation that higher rank DT invariants can be 
expressed in terms of rank 1 DT invariants and “point contributions” Note that unlike 
the rank 1 case, there are no signs in this formula (essentially because rank 2 is even). 
Comparing to ([^ and Theorem A, Conjecture C suggest DT(C) = e(C), but this is not 
the case. In fact, DT(C) can be zero due to obstructions (Example 6.5). The following 
is strong evidence for Conjecture C. Let Tq C T denote the “Calabi-Yau torus” dehned 
by tit 2 t 3 = 1. Let C C Aiycx be a compact connected component of the To-hxed locus 
with constant reflexive hulls TZ and cokernels of dimension 0. Using To-localization and 


®Another approach to DT invariants of Y is by using “framing at infinity” as in the work of D. Oprea 
|Opr| . In this case, the generating function becomes a product of rank 1 generating functions by using 
an extra torus action on the framing. In our theory, such a splitting does not occur due to the presence 
of singularities in the reflexive hull. 

^^Mentioned to us by D. Maulik and J. Manschot. 
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assuming analogs of two conjecture^ used in the To-localization of stable pair theory of 
Pandharipande-Thomas |PT2j . we prove 

Y, DT(C)= Y e(C.) = e(C). 

CiCCC* CiCCC* 

Here the sum is over all connected components of the T/Tq = C*-fixed locus of C. 
See Remark 6.25 for the precise statement. Combinatorially, this means that whenever 
DT(Cj) = 0 for some Cj, other “buddy components” Cj in the same C*-fixed locus will 
compensate to recover the Euler characteristic. 

Main Conjecture 6H is a generalization of Conjecture 0 to arbitrary smooth toric 
Calabi-Yau 3-folds and Chern classes. It expresses generating functions of rank 2 DT 
type invariants in terms of Euler characteristics and signs. The signs only occur in 
the presence of legs, i.e. when the cokernel is 1-dimensional. We provide the following 
evidence for Main Conjecture 6.1[ 


• Explicit examples (Section 6.1, 6.2). 

• A proof in the case of “expected obstructions” (Theorem 6.17). 

• A proof using Tn-localization based on the analogs of two conjectures of IPT2I 
(Theorem [R^. 

Notation. For any sets A, B, we denote projections to the factors hy pa '■ A x B ^ A 
and pb ■ A X B ^ B. If A, B are schemes with coherent sheaves S,fF, we write 
£ ^ fF := p*a£ ® We denote the closed points of a finite type C-scheme B by B^- 


Acknowledgements. We would like to thank K. Behrend, J. Bryan, J. Manschot, 
D. Maulik, and R. P. Thomas for useful discussions. We would also like to thank the 
anonymous referees whose comments led to an immense improvement of the exposition, 
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EP/G06170X/1, “Applied derived categories” (while at Imperial College), a PIMS post¬ 
doctoral fellowship (CRG Geometry and Physics, while at UBC), and NWO-GQT and 
Marie Sklodowska-Curie Project 656898 (INVLOCCY) (while at Utrecht). 


Part I: Classical 


2. Fixed loci 


In this section we recall Klyachko’s description of T-equivariant torsion free and re¬ 
flexive sheaves on a smooth projective toric 3-fold X. In the rank 2 case, we use it to 
describe (scheme-theoretically) the T-fixed locus of the moduli space of /i-stable torsion 
free sheaves and deduce it is smooth (Theorem 2.10). We also give a combinatorial de¬ 
scription of the fixed locus of the Quot scheme of a T-equivariant rank 2 reflexive hull in 
terms of new combinatorial objects called double box configurations (Definition |2.17[). 


2.1. Toric 3-folds. Let X be a smooth projective toric 3-fold with dense open torus T. 
We recall some basic facts and notation from toric geometry. For a general introduction 
to toric varieties see [Fulj . Let A(X) be the Newton polyhedron of X determined by a 
polarization (e.g. |MNOPl1 Sect. 4.1]). Note that we are not actually fixing a polarization 

^'^One conjecture asserts smoothness of C. 
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at this stage, so A(X) does not come with an embedding in the character lattice. Denote 
the collection of vertices of A(X) by D(X) and the collection of edges of A(X) hy E{X). 
The vertices a G V{X) correspond bijectively to the cones of dimension 3 of the fan of 
X. We denote the edge between two adjacent vertices a,/3 G V{X) by a(3 G E{X). The 
edges a/3 G E{X) correspond bijectively to the cones of dimension 2 of the fan of X. We 
denote by Ua = the T-invariant affine open subset corresponding to a. Furthermore, 
we write X^ for the fixed point corresponding to a and Cap = for the T-invariant line 
joining Xa and Xp. The toric divisors are labelled by the faces of A(X). We denote the 
collection of faces of A(X) by E{X) and label its elements by p G T(X)g The toric 
divisor corresponding to p G E{X) is denoted by Dp. 

For any a G D(X), there are coordinates {xi,X 2 ,x^) on Ua = and t = (ti,t 2 ,t 3 ) oii 
T = such that 

t- {Xi,X3,X3) = {tiXi,t2X2,t3X3)- 

In these coordinates, the fixed point Xa has cotangent representation 

{dxi)c ®ti + (dX2)c ®t2 + {dX 3 )c ® ts, 
where (-/c denotes C-span. We therefore have a T-equivariant isomorphism 

(5) Kx\ua — ®tlt2t3- 

For each aid G E{X), there exist integers map, m'^^ such that the normal bundle Nc^p/x 
is given by 

^Ccp/X — 0^i{map) © 0^i{m'ap)- 

The transition function between the charts Ua, Up is given by 

( 6 ) {Xi,X2,X3) ^ {x'l^ ,X2X'1'^°‘^ ,X3X^'^"‘^), 

where Cap is defined by X 2 = x^ = 0. Here we follow the notation of |MNOPl] . 


2.2. Equivariant torsion free sheaves. We give Klyacho’s description of T-equivariant 
torsion free sheaves on a smooth projective toric 3-fold X |Klyl Kly2| . Klyachko’s work 
was further developed by M. Perling |Perlj and we will mostly follow his notation. Most 
constructions of Sections 2.2- 2.4| work for X of any dimension, but we stick to dimension 
3 for notational convenience. 

Let X be a T-equivariant quasi-cohere nt sh eaf on X and a G V{X). Let {xi,X 2 ,X 3 ) 
= as in Section 


be coordinates on U^ 
global section functor 


2.1 


We first study the restriction E\u^- The 


gives an equivalence between the categories of quasi-coherent 
sheaves on Ua and C[a;i,X 2 , Xsj-modules. It is not hard to extend this to an equivalence 
between the categories of T-equivariant quasi-coherent sheaves on Ua and C[xi,X 2 ,x^- 
modules with regular T-action |Perlj . 

Given a T-equivariant quasi-coherent sheaf Ea on Ua, we have a decomposition into 
weight spaces according to the character group X(T) = I? 


H\Ea)= 0 Ea{h,k2,h). 

The sheaf Ea is coherent if and only if H^{Ea) has a finite number of homogeneous 
generators. Moreover Ea is torsion free if and only if the multiplication maps xi,X 2 ,X 3 


11 


We identify F{X) with the collection of rays of the fan of X by sending a face to the ray it is 


transverse to. 
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are injective on all weight spaces -Fa(A;i, ^ 2 , ^ 3 ) |Perlj . Therefore, the category of T- 
equivariant rank r torsion free sheaves Ta on 1]^ is equivalent to the category of collections 
of complex vector spaces {Fa{ki, ^ 2 , k^)} satisfying: 

• The vector spaces Fa{ki, ^ 2 , ^ 3 ) form a triple hltration 

Fa{ki, k2, ks) G Fa{ki + 1, k2, k^), 

Fa{ki, k2, ks) C Fa{ki, k2 + 1 , ^ 3 ), 

Fa{ki, k2, k^) C Fa{ki, /C2, ^3 + 1)- 

• For all ki, ^ 2 , ^3 S> 0 we have Fa{ki, k 2 , k^) = C'’. 

• Whenever fci 0 or ^2 ^ 0 or /C 3 0 we have Fa{ki, ^ 2 , ^ 3 ) = 0. 

We refer to {Fa{ki, k 2 , ^ 3 )}(fci,fc 2 A 3 )ez 3 as a rank r cr-familyj^ The maps in the category 
of T-equivariant rank r torsion free sheaves on Ua are T-equivariant morphisms and the 
maps in the category of rank r cr-families are endomorphisms of compatible with the 
triple hltrations. 

A T-equivariant torsion free sheaf T” on X gives rise to T-equivariant torsion free 
sheaves Fa ■= F\ua and corresponding a-families 

F := {TQ,(fci,/c25 fc3)}(fcl,fc2,A:3)6Z3,aeU(x)• 

Conversely, given such a collection of cr-families, when do the corresponding sheaves Fa 
glue to a sheaf F on XI Note that for hxed ^ 2 , ^ 3 , the vector space Fa{ki, ^ 2 , ^ 3 ) C 
does not change for ki S> 0. We denote this “limiting vector space” by Fa{oo, k 2 , k^). 
It is not hard to show that the gluing conditions are given by the following equations 
IKI^IPctT] 

(7) Fa{oo,k2,k3) = Ffi{oo,k2,k3), 

for all a, (3 E V{X) and ^ 2,^3 G Z. Here the coordinates are chosen such that Cap is 
dehned by equations 0:2 = 0:3 = 0 in Ua and x '2 = Tg = 0 in Up. This describes an 
equivalence between the category of T-equivariant rank r torsion free sheaves on X and 
collections of rank r cr-families satisfying the gluing conditions ([^ |Perlj . The maps in 
both categories are the obvious ones. 

Definition 2.1. Let T” be a T-equivariant torsion free sheaf on a smooth projective 
toric 3-fold X with corresponding collection of cr-families F. We dehne the characteristic 
function Xt -F as follows 

Xx ■— {Xx}a£ViX), 

—)■ Z, Xxiki, k 2 , ^ 3 ) := dim(T«(fci, ^ 2 , ks)). 

We denote the collection of characteristic functions of T-equivariant torsion free sheaves 
on X by X. 

Characteristic functions of members of a flat family of T-equivariant coherent sheaves 
are locally constant over the base |Koolj . One can construct projective coarse moduli 
spaces of T-equivariant torsion free sheaves with hxed characteristic function using GIT. 
Moreover, one can choose a GIT stability condition for which GIT stability and Gieseker 
stability coincide |Koolj . 

Let X be a T-equivariant torsion free sheaf on X. Then there is an explicit formula 
for the Ghern character ch(X) in terms of the characteristic function Xx- This formula 

^^This terminology is due to Perling [Perl] . Here a denotes the cone of maximal dimension corre¬ 
sponding to vertex a. In our notation a-family would perhaps be better terminology. 
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is due to Klyachko |Klyl Kly2| . An alternative way for obtaining such a formula is 
by T-equivariant devissage. Concretely, this works as follows. The cohomology ring 
is explicitly determined by the fan (or polyhedron) of X. It is the free Z- 
module generated by the T-invariant closed subsets 


modulo certain relations m- For example Xa = Xp for all a,/? G V{X) is such a 
relation. There are three linear relations among the toric prime divisors Dp and certain 
relations among the Ca/s- The T-equivariant Picard group Pic^(X) is generated by Dp 
without relations. Inside the T-equivariant iP-group Kq{X), there is an explicit way of 
writing T” as a hnite sum of elements of the form 


Od, ® L, Oc^p ® T, Ox^ ® T, 


for L G Pic^(X). The algorithm for getting such a decomposition is explicitly described 


m |GK1] for the case X = P^, r = 2, and D reflexive. See Example 2.3 below for 
the equations. It is straightforward to generalize the algorithm to any X and T” a T- 
equivariant rank r torsion free sheaf on X. For any % G df, we write ch(x) for the Chern 
character determined by a characteristic function %. Characteristic function determines 
Chern character and hence Hilbert polynomial (by Hirzebruch-Riemann-Roch). For a 
hxed Chern character ch, we denote by 


A’ch C A 


the collection of characteristic functions % satisfying ch(x) = ch. 


2.3. Equivariant reflexive sheaves. Let X be a smooth projective toric 3-fold with 
polarization H. Recall that a coherent sheaf T” on X is /x-semistable (resp. /x-stable) if 
and only if D is torsion free and we have /i^: < /xjr (resp. /i^: < /ijr) for all subsheaves 
8 C D with 0 < rk(T) < rk(T') |HL1 Def. 1.2.12]. Here the slope /rjr is dehned by 

CiH^ 

Recall from the introduction that we always work on the open subset of /i-stable sheaves 
and disregard strictly /i-semistable sheaves completely. A torsion free sheaf X on X is 
/i-stable if and only if its reflexive hull D** is /i-stable|^ Klyachko |Kly2| found out 
that T-equivariant reflexive sheaves on X have a particularly straightforward descrip¬ 
tion. Moreover, their stability is easily described in terms of their characteristic function 
|Koolj . We will now describe this. 

We denote by I the number of faces of the polyhedron A(X). Note that I equals 
the rank of Pic^(X). Klyachko |Kly2| gives an equivalence between the category of 
T-equivariant rank r reflexive sheaves on X and the category of collections of flags 


R {Rp{k)}k&'L,p&F(x) 

■ ■ ■ C Rp{k - 1) C Rp{k) C Rp{k + 1) C ■■■ 


satisfying 

• Rp{k) = 0 for all /c -C 0, 

• Rp{k) = for all /c S> 0. 


^^The reason is that for all £ C A the sheaves £ and £** 
subset of codimension > 2 so U£ = US** ■ 


are equal on the complement of a closed 
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The morphisms of both categories are the obvious ones. 

We now describe how a family of flags R gives rise to a T-equivariant reflexive sheaf 
TZ on X. Let a G V{X) and let Ua = be the corresponding T-invariant affine open 
subset. Write the coordinates on Ua by {xi,X 2 ,X 3 ) as in Section 2.1 
divisor Xi = 0 corresponds to ray pi G F{X). Define vector spaces 


Suppose the toric 


Ra{ki, k2, ks) := Rp^{ki) fi Rp^{k2) n Rp^iks). 


Then 


{Ra{ki, k2, /c3)}(fci,fc2,fc3)ez3,Q:ey(x:) 

is a collection of rank r a-families satisfying the gluing conditions ([^. The corresponding 
sheaf 7^ on X is reflexive |Kly2 IPer2j . Roughly speaking the reason is that a reflexive 
sheaf is determined by its restriction to the complement of any closed subset of codi¬ 
mension > 2 |Har2[ Prop. 1.6] and the flags Rp^{k), Rp^{k), Rp^{k) precisely correspond 
to the restriction of the reflexive sheaf to C x C* x C*, C* x C x C*, C* x C* x C. 

In the case r = 2, a family of filtrations R is entirely determined by the integers where 
the dimensions jump together with a choice of 1-dimensional subspaces for the jumps. 
More precisely, for each p G F{X), there are unique Up G Z, Up G Z>o and Pp G such 
that 


Rp{k) = 


0 

Pp 


for all k < Up 
for all Up < k < Up + Vp 
for all Up + Vp < k 


where Pp does not occur when Vp = 0. 


Definition 2.2. Let TZhe a T-equivariant rank 2 reflexive sheaf on X corresponding to 
a collection of filtrations {Rp{k)}. We refer to the data 

{{Up,Vp,Pp)}p^F(X) 

described above as the toric data of TZ and abbreviate it by (u, v,p). Given toric data 
(u, V, p), we dehne 

:= dim(pp ripp/) G {0,1}, 

Sp,p,p” ■= dim(pp ripp/ npp//) G {0,1}. 

We abbreviate the collection of these number by <5. Clearly the the numbers (u, v,5) 
determine the characteristic function Xn ci-nd vice versa. We treat both notions on equal 
footing. 

Example 2.3. The toric 3-fold X = P^ is described by the lattice N = 1? together with 
the fan consisting of the 3-dimensional cones 

CTi = (ei, 62 , e3)z>o, 

<^2 = (^ 2 , 63, —Cl — 62 — 63)2^15, 

<73 = (ei, 63, —61 — 62 — e 3 )z>o, 

(Ji = (ci, 62, —61 — 62 — e 3 )z>o. 

Here ( 61 , 62 , 63 ) is the standard basis of I?. We denote the rays generated by 61 , 62 , 
63, —61 — 62 — 63 by pi, p2, P3, pA- The corresponding polyhedron is a tetrahedron 
and F{X) = {pi,P 2 ,Ps,^ 4 }- Let h denote the hyperplane class on X and let 7^ be a 
T-equivariant rank 2 reflexive sheaf on X described by toric data (u, v,p). We dehne 
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Ui := Mp,, Vi := Vp^, 6ij := and Sijk := In |GKlj the Chern classes of TZ are 


computed using T-equivariant devissage as mentioned in Section [272 
ci{Ti) = + y^Vi)h, 

i i 

1 


c,(n) = \cdnr + - 26 ,,) 


i<j 


- 4 




CsiTl) = ^ ViVjVkil 
i<j<k 


^ij ^ik ^jk • 


This expresses ch(7?.) in terms of the characteristic function Xn- By further removing 
Chern characters of Oc^p®L, Ox^ with L G Pic^(P^), we obtain explicit expressions 
for ch(J^) in terms of Xf given T-equivariant rank 2 torsion free sheaf ZF on P^. 


Back to any smooth projective toric 3-fold X. The following proposition is very useful. 


Proposition 2.4. Let TZ be a T-equivariant rank 2 reflexive sheaf on X with toric data 
(u, v,p). Let a G V{X) and denote by pi,a, P 2 ,a, P 3 ,a £ the faces sharing vertex a. 

Then TZ\uc, is singular, i.e. not locally free, precisely if Vp^^a, Vp^^a, Vp,^^a are all positive 
and pi^ot, P2,a, P3,a are mutually distinct. If IZlu^ is singular, then the length of its 
singularity equals the product Vp.,^^aVp 2 ,aVp 3 ,a- Moreover, the degree of cfllZ) equals the 
sum of the lengths of the singularities of {TZ\u,„}aev(x)- 


Proof. For X = P^ this is immediate from |Har2i Prop. 2.6] and the formula for cflIZ) of 
Example 2^ For general X, the proof is easily adapted as is discussed in detail in the 
proof of |GK11 Prop. 3.6]. □ 


Let 7^ be a T-equivariant rank 2 reflexive sheaf on X described by toric data (u, v, p). 
Like in the case of Chern classes, /i-stability of TZ can be described in terms of the toric 
data only. More precisely TZ is /x-stable if and only if for any 1-dimensional subspace 
g C C2 

dim{ppnq){DpH^)vp<^ Y iDpH^)vp, 
p&F{X) peF(X) 


where we recall that H denotes the polarization and Dp the toric divisor corresponding 
to p G F{X) (Section |2.1 ). A similar description exists for any rank on any polarized 
smooth projective toric n-fold |Koolj . 

Let X be a T-equivariant torsion free sheaf on X. Then X is /i-stable if and only 
if its reflexive hull TZ ;= T** is p-stable. So in order to understand p-stability of such 
sheaves, we only need to write the family of hltrations R corresponding to TZ in terms of 
the collection of cx-famihes F corresponding to X. This can be done as follows: for any 
a G iy(X) denote by p\, p 2 , ps the faces sharing vertex a, then 

Rpi{k) = Fo,{k,oo, oc), Rp^ik) = Fa{oo,k,oc), Rp^i^k) = Fa{oo,oo,k), 
for all A: G Z ( |Kly2| , see also ISKH). 


2.4. General structure of fixed loci. Let x G X be the characteristic function of a 
T-equivariant rank 2 torsion free sheaf on X. There exists a natural moduli functor of 
families of /i-stable T-equivariant torsion free sheaves on X with characteristic function 
X IKoolj g Using the previous section, a fairly straight-forward GIT construction yields 


^■^The paper |Kool| rather deals with Gieseker stability. The case of ^-stability is easier. 
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a quasi-projective C-scheme corepresenting this functor. In this section we describe the 
outcome. For details and proofs (in a much more general setting) see |Koolj . 

For a hxed characteristic function x = {Xa}a£V{x) £ df, the following dehnitions 
identify connected components in the domain of each Xa where Xa takes value 1. To 
each such connected component, we will then associate a moduli factor (up to gluing). 

Definition 2.5. Denote the Euclidean norm on by || ■ ||. Let x = {Xa}aev(x) ^ df 
and consider Xa- Let S C X{T) = then S is called a connected component of I’s 

associated to Xa when 

Xa{ki, k 2 , fcs) = 1, for all (fci, fca, ^a) e S 

for all {ki, k 2 , k^) G S and (/i, I 2 , 13) G : 

if Xa{h,h,h) = 1 and ||(fci, ^ 2 , ^s) - (^i,^2,^3)|| = 1 then {h,l 2 ,h) e S, 

and S does not contain a non-empty proper subset with these properties. 

Definition 2.6. Let x = {Xa}aevix) ^ df and £x a G V{X). Consider a connected 
component of I’s, k, associated to Xa- 

• If A is bounded in Z^, then we refer to it as a vertex component associated to Xa- 

• The connected component k is called a local face component associated to Xa if 
it satishes the following property. There exists a face p G F{X) containing the 
vertex a and with Vp > 0 such that if we choose coordinates for which the toric 
divisor Dp corresponds to {xi = 0 } and write Ui := Up, Vi := Vp, then we have 
(fci, ^ 2 , k^) G K for all integers Ui < ki < Ui + vi and k 2 , k^ ^ 0. 

• In all other cases n is called a local edge components associated to Xa- 

Definition 2.7. Let x = {Xa}a&v{x) G X. The toric data (u, v,p) of any reflexive hull 
of a T-equivariant torsion free sheaf on X with characteristic function x Las the same 
values u, v. Consider all connected components of I’s of all {Xa}a£Vix) at once. It is 
clear that connected components of I’s associated to Xa, Xp glae just like a-families ([^. 
Vertex components never glue, but local face or local edge components associated to one 
Xa Biay glue to local face or local edge components associated to another xp- Note that 
local face components may glue to local edge components. After gluing, we refer to a 
global connected component of I’s containing a local face component as a face component 
associated to X- All other global connected components of I’s which are not vertex or 
face components are called edge components associated to X- We denote the number of 
face, edge, vertex components associated to x by a, b, c. 

Let the situation be as in the previous dehnition. To each connected component of I’s 
of X we associate a choice of C C C^. This specifies a hltration {Fa{ki, k 2 , k 3 )}a^vix), 
i.e. a T-equivariant torsion free sheaf on X with characteristic function x- Hence the 
closed points of 

(pl)a ^ ^ ^pl^c 

are in bijective correspondence with all possible collections of cr-families of T-equivariant 
torsion free sheaves on X with characteristic function x- Moreover, two points corre¬ 
spond to isomorphic objects if and only if they lie in the same SL(2, C)-orbit. Here 
SL(2,C) acts by matrix multiplication on each factor P^. The SL(2, C)-equivariant line 
bundles on this space are in 1-1 correspondence with elements of |Dolj 


X X Z ^ 
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via the correspondence 

({«.}?.!. K}Li) « HtiOp.(a,) H H HJ.,Op.(cO. 

We want to pick an SL(2, C)-equivariant line bundle for which the properly GIT stable 
points exactly correspond to the collections of a-families for which the associated sheaf 
is /i-stable. Using the previous section, one can show that the following choice works 
|Koolj 


tti ;= '^{H^Dp)vp, bj = 0 , Ck = 0 , 
p 


where the sum is over all faces p G F{X) for which Up > 0 and the corresponding local 
face component is contained in the (global) face component indexed by i. The stability 
does not depend on the edge and vertex components but only on the face components. 
In particular, we get an open subset Uxix) C (P^)“ such that the closed points of 

U§{x) X (P')'x (Pi)'= 

precisely correspond to /i-stable sheaves. Unfortunately the above SL(2, C)-equivariant 
line bundle is not ample. For ampleness, all Oj, Cj must be positive. Therefore we £x 
i? S> 0 and take 

tti := R'^{H‘^Dp)vp, bi = 1, c* = 1, 

p 

where the sum over p is as above. Then the properly GIT stable locus is still 

Uf(x) X (Pi)'x (Pi)'= 


and the GIT quotient 


■■= U^{X) X (P')' X (Pi)^ / SL(2,C) 


is a quasi-projective variety whose closed points exactly correspond to the T-equivariant 
isomorphism classes of T-equivariant rank 2 /i-stable torsion free sheaves on X with char¬ 
acteristic function x- The space Xtxix) is projective when gcd(2, ciH"^) = 1. Moreover, 
M-xix) is smooth, because Uxix) x (P^)^ x (P^)'^ is smooth and PGL(2, C) acts without 
stabilizers. One can show that A4x(x) indeed corepresents a natural moduli functor as 
follows |Koolj : 


Theorem 2.8. Let X be a smooth projective toric 3-fold with polarization H. Let x G T 
he the characteristic function of a T-equivariant rank 2 torsion free sheaf on X. Then 

M ^( X ) := Uf (x) X {Fy X (P')'= / SL(2,C) 

is a smooth quasi-projective variety corepresenting the moduli functor of families of T- 
equivariant p-stable torsion free sheaves on X with characteristic function X- 


Remark 2.9. In the above theorem, the double dual map is given by 

U^X) X (P')^ X (P')^ / SL(2,C) f/f (x) / SL(2,C), 

where we denote the latter quotient by Nffix)- We deduce that if Xtxix) 7^ then 
a > 3 (or else the connected component of reflexive hulls J^x ix) = 0)- We also deduce: 
all closed points of Mx(x) have the same reflexive hull if and only if a = 3. Suppose 
this is the case. Then Xtxix) — (P^)^ x (P^)'^. Moreover, Mx (x) is an isolated reduced 
point {[7^]} of the T-£xed locus of the moduli space of /r-stable reflexive sheaves on X. 
Therefore 


Ext^(7^,7^)'^ = 0. 
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We have a tautological line bundle C>pi(l) on each factor of Mx(x)- Using the 
tautological inclusions 

Opi (—1) c Opi © Ojpi, 

it is easy to construct a universal family F on Aixix) fitting in a short exact sequence 

0 —^ F —^ p*xn —^ Q —^ 0, 
where the cokernel Q is Wl;^(x)-fiat. 


Suppose now X is a smooth projective toric 3-fold with polarization H and let M.x '■ = 
Cl, C 2 , C 3 ) denote the moduli space of rank 2 p-stable torsion free sheaves on X 
with indicated Chern classes. We consider its fixed locus Denote by df( 2 ,ci, 02 , 03 ) the 

collection of characteristic functions of T-equivariant rank 2 torsion free sheaves on X 
with Chern classes Ci, 02 , 03 . Forgetting the T-equivariant structure defines a map 


( 8 ) 


U ^xbd^M^x 

XSA'(2,ci,C2,C3) 


It is not hard to show this is a morphism |Kool] . Any element of AdJ admits a T- 
equivariant structure, which is unique up to tensoring by a character of X{T) |Koolj . 
The map (|^ is therefore surjective but not injective. It can be made injective as follows. 
Choose any vertex ao G V{X) and denote by pi,p 2 ,P 3 £ -U(X) the faces having ao as a 
vertex. Define 


jv^slice 

‘^( 2 , 01 , 02 , 03 ) 


C Xi 


(2,01,02,03) 


as the subset of characteristic functions x which the corresponding (u, v) satisfy 


(9) 

Then 


«P 1 = «P 2 = «P 3 = 0 - 


U 




X 


^gV^lice 


( 2 , ci , C 2 , c 3 ) 

is a biiective morohism. In fact, it is an isomorohism of schemes |Kooll Cor. 4.io]g 
Combining with Theorem |2.8| gives: 


Theorem 2.10. Let X be a smooth projective toric 3-fold with polarization H and let 
M.X '■= A 1 x (25 Cl) C 2 ; ©)■ Then the forgetful map described above is an isomorphism of 
schemes 


■^x= U -^xix)- 




slice 

( 2 , ci , C 2 , C 3 ) 


In particular, Mx is a smooth quasi-projective variety. 


2.5. Quot schemes of reflexive sheaves. As discussed in the introduction, double 
duals of members of a flat family of torsion free sheaves do not need to form a flat 
family. However, a result of Kollar |Kol] implies the double dual map is constructible 

(■) •-^x(2) Cl, C2, C3) > I_I (2, Cl, C2, C3). 

At the level of closed points, the fibre over [7^] G Nffifl, ci, C 2 , Cg) is the Quot scheme 

QuotJ^(7^, C 2 , Cg) 

^^This construction works for any rank r and any smooth projective toric n-fold. 
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of quotients of TZ of dimension < 1, such that 

C 2 — C2 + C 2 
C3 = C3 + C3 + CiC2h 


( 10 ) 


In this section, we develop a way of indexing the connected components of Quot;((-(7?., C 2 , Cg)^ 
by a new type of combinatorial objects called double box conhgurations. Each connected 
component is isomorphic to a product of P^’s as we will show Erst. 

Suppose TZ is any T-equivariant rank 2 reflexive sheaf on X described by toric data 
(u,v,p) (Dehnition 2.2). Denote by X(."^c”{TZ) C X the collection of characteristic 


functions of T-equivariant subsheaves of 7Z such that the cokernel has Chern classes d 


^2i ^3 


Ji\T 


2i 


are T-invariant submodules of 


C 3 . The closed points of Quotj('(7?., C 2 

{H\Uo.,n)}^^vix) 

satisfying the gluing conditions ([^. This gives a decomposition into connected compo¬ 
nents 

( 11 ) 


QuotJf(7^,C2,C3)^ = IJ C^. 




Proposition 2.11. EcichC-^ of 0 is isomorphic to a product o/P^ ’s, where the number 
o/P^ ’s is the number of vertex and edge components associated to X- 


Proof. This can be proved using the methods of |Koolj . In fact, this case is much easier 
than Theorem |2.10[ Step 1: introduce a moduli functor of T-equivariant families of 


quotients of 7Z with Chern classes 02,03 as in |Kool| Sect. 3.1]. Step 2: describe these 
families via a family version of Klyachko’s hltrations like in |Kool( Sect. 3.2]. Step 3: 
construct the moduli spaces C-^ as GIT quotients and show they corepresent the previous 
moduli functors |Kooll Sect. 3.3]. Step 4: lift the action of T on X to QpiotxiJZx'ix'i) 
(done in the proof of |Kooll Prop. 4.1]). Step 4: show the forgetful morphism is an 
isomorphism. This goes similar to |Kooll Sect. 4.3] but much easier: this time elements 
of QuotjY(7^, C 2 , Cg)^ admit a unique T-equivariant structure. □ 

Remark 2.12. Note that we do not assign factors of P^ to the face components in 
This is because the reflexive hull TZ is kept fixed so does not have 


Proposition |2.11 
moduli. 


Fix a G V{X) and TZa '■= 'hZ\ua.- Our goal is to give a combinatorial description of 
characteristic functions y of rank 2 T-equivariant subsheaves of TZa- We denote the 
collection of such characteristic functions by XiflZa). Let pi,p 2 ,P 3 ^ -^(W) be the faces 
with vertex a. Suppose the labeling is chosen such that the toric divisor Dp. in chart 
Ua corresponds to the coordinate hyperplane {x* = 0}. Then y gives rise to integers 
(Dehnition |2.2[ ) 

Ui . G 

Vi := Upi > 0. 

Moreover, TZa is described by toric data (Dehnition |2.2[ ) 

Ui G Z, 

Vi > 0 , 

Pi := Pp. G P^ 
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Figure 1. Toric data Ui,Vi,pi lying in the character lattice X{T) = 1?. 
There are three local face components with weight spaces of dimension 1: 
the green (generated by pi), blue (generated by P 2 ), and yellow (generated 
by P 3 ) regions. The red region has weight spaces of dimension 2 . All other 
points of the character lattice have weight spaces of dimension 0 . 


This data is represented by Figure 1. The sheaf 71^ is singular if and only if vi, ^ 2 , 'i '3 > 0 
and pi,P 2 ,P 3 are mutually distinct (Proposition |2.4[ ). We discuss the case TZa is singular. 
The (easier) case TZa is locally free is treated in Remark 2.26 In what follows, we let 


TZap := 7Z 




where Uap = Ua7}Up for all a/3 G E{X). Recall that (Section |2.3[ ) 

Ra/li (^2, h) = Raioo, /C2, ks) 
for all /c 2 , ks E Z (and similarly for 7ZaP2^ TZap^)- Note that 


RaPi{k2, k^) 


for all k 2 >U 2 + V 2 and /cs > M 3 + M 3 
P 2 for all ^2 < ^2 < M 2 + V 2 and ^3 > Ms + M 3 
P 3 for all k 2 >U 2 + M 2 and M 3 < ^3 < Ms + M 3 
0 otherwise. 


In this way TZapi determines the integers M 2 , M 3 , M 2 , M 3 (and similarly for TZap 2 -i R-aPs)- 
We start with some dehnitions. Given a point k = [ki, ^ 2 ) in we write 

G(k) := {(fci + h, ^2 + h) I h £ lP>o}- 


Definition 2.13. A 2D partition based at (mi, m2) G is a subset A C satisfying the 
following condition. If {ki,k 2 ) G A then ki > mi, ^2 > 1^2, and (/i,fc2) ^ A, (fci,/2) G A 
for all mi < /i < fci, m 2 < ^2 < ^ 2 - We call A finite if |A| < 00 . 





















RANK 2 SHEAVES ON TORIC 3-FOLDS 


17 


Definition 2.14. Let = (1;2,3), (2; 1,3), or (3; 1,2). Consider a triple of 

finite 2D partitions A = (Ai,A 2 ,A 3 ) based respectively at {ui'^Uin +^ 4 //), (wj/ +Vii,Uiii), 
{ui! + Vii^Uiii + Vin) and subject to the following condition. Let Ci := C{uii,Ui» + Vi»), 
C 2 := C{uii + and D := C{uii + Vi’^Uin + n*")- Define 

Ain •= Ai n A 2 n A 3 , 

Aout := (Ai U A 2 U A 3 ) n D. 

Condition-, each element of Aout \ A^ belongs to exactly two of the partitions Ai, A 2 , A 3 . 
We define an equivalence relation ~ on the collection of such triples A = (Ai, A 2 , A 3 ) as 
follows. Write A ~ whenever: 

(1) \j n {Cj \ D) = fXj n {Cj \ D) for all j = 1, 2, 

(2) Ain = Min) 

(3) Aout = Mout- 

We refer to such equivalence classes as double square configurations in TZa/Si and denote 
the collection of such equivalence classes by KilZajsfi. For any A G A( 7 ^o/ 3 i), define 
the size |A| as 

3 

1^1 •= (5^|A.|) -|Aout|. 
i=l 

Note that |A| is independent of the choice of representative of the equivalence class. 

Example 2.15. Let (mi,M 2 ) = (0,0) and (^ 1 ,^ 2 ) = (3,1). Consider the following triples 
of 2D partitions 

(^, m, □), (^, &, 0), (m, &, □). 

For each of these triples A, we have that Ai fl (Ci \ D) = A 2 H (C 2 \ D) = m, 

Ain = and Aout = Therefore all three triples are equivalent and determine the 
same double square configuration. 


Given a point k = {ki,k 2 , kfij in we write 

C'(k) := {{ki + /i, k 2 + I 2 , ks + I 3 ) \ k G IF>o}. 

Referring to Figure 1, we are interested in the following cones 

Cl := C(Mi, U2 + V2, U3 + V3), 

C 2 := C{Ui+Vi,U2,U3 + V3), 

C3 := C{ui + vi,U2 + V2, M3), 

D -.= C(mi +Vi,U2 + V2, U3 + M3). 

For all {i-,i',i") = (1; 2, 3), (2; 1, 3), (3; 1, 2), let -)■ Pi{ki,k 2 ,k 3 ) 

denote projection. 


(fcj', ki ") 


Definition 2.16. A 3 D partition based at {mi, m2, m3) G is a subset tt C satisfying 
the following condition. If (/ci,/c2,^3) G tt, then ki > mi, k2 > m2, k3 > m3, and 
{Ii,k 2 ,k 3 ) G TT, {ki,l 2 ,k 3 ) G TT, {ki,k2,l3) e TT for all mi < /i < ki, m2 < k < /^2, 
7R3 < ^3 < ^3- We call TT finite if | 7 r| < 00. 


Definition 2.17. Fix outgoing double square configurations A* G A{7lai3i) for all i = 
1 , 2 , 3 . Consider a triple of (not necessarily finite) 3 D partitions tt = (771,772,773) based 
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at (mi,M 2 + V 2 ,U 2 , + P 3 ), {ui + + 'P 3 ), (mi + Pi,M 2 + ^ 2 , 113 ) subject to the two 

conditions described below. Define 

TTin := TTi n 712 n TTs, 

TTout := (tti U 712 U tts) n D. 

Take S> 0 sufficiently large such that the following projections do not depend on N 

Ai(7ri) := Pi(7ri \ + iV] x Z^), 

Ai(7r2) := Pi(712 \ [ui,ui + iV] X I?), 

Ai( 7 i 3 ) := Pi (713 \ [ui,Ui + iV] X Z^), 

A2(7ri) := P2(7ri \ Z x [^ 2 , U 2 + N] x Z), 

A2(7l2) := P2(7l2 \ Z X [U2, U 2 + N] X Z), 

A2(7r3) := -^ 2(713 \ Z X [U2, U2 + N] X Z), 

A3(71i) := P3(71i \ Z^ X [U3,U3 + N]), 

A3(7I'2) := -^ 3(712 \ Z^ X [ 113 , M 3 + iV]), 

•^3(7I'3) •= \ Z^ X [m3, M 3 + iV]). 

Condition 1 : Each element of Tiout \ T^in belongs to exactly two partitions Tij. 

Condition 2\ 

(Ai(7i2), Ai(7i3), Ai(7ri)) ~ Ai, 

(A2(7ri), A2(7l3), A2(7r2)) ~ A 2 , 

(A 3 ( 7 ri), A 3 ( 7 l 2 ), A 3 ( 7 r 3 )) ~ A 3 . 

We define an equivalence relation ~ on the collection of such triples as follows. Write 
71 vj whenever: 

(1) Tij n (Cj \ P) = cuj n {Ci \ D) for all z = 1, 2, 3, 

(2) Tljn 

(3) 71 out ^ out • 

We refer to such equivalence classes as double box configurations in TZa and denote the col¬ 
lection of such equivalence classes by n(P„, -^ 2 , '^ 3 )- For any ti G n(P„, ^ 1 , -^ 2 , -^ 3 ), 

we define the size of ti as 

3 

17r I . ^ ^ ^ I I ^ 171 out 1 5 

i=l 

where | ■ | on the RHS denotes the renormalized volumeNote that | 7 i| is independent 
of the choice of representative of the equivalence class. 

The purpose of this definition is to give a new description of the characteristic function 
Xa of a rank 2 T-equivariant torsion free sheaf Pq C TZa on Ua- Since 

XTc < Xt** = 

the difference xHa ~XTa indicates regions of dimension 0,1,2. Double box configurations 
are a combinatorial way of indexing all possible differences that can occur in this way. 
The precise statement is: 

^®The renormalized volume of a 3D partition tt is defined as |7r| = ~ containing □). 

See |1V1N()P1| for details. 
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Proposition 2.18. Let TZa be a T-equivariant rank 2 reflexive sheaf on toric chart 
Ua = LetTZaiSi •= ’^a|c*xCxC; "^ 0/32 •= ’^a|cxC*xC; ondlZaps '■= TZa\cxCxC* ■ Denote 
by XiJZa) the collection of characteristic functions of T-equivariant rank 2 torsion free 
sheaves on Ua with T-equivariant reflexive hull TZa- Then there is a natural bijection 

t^iJZa) — I_I n( 7 ?.Q,, Ai, A2, A3). 

Ai G A(7^c/3l) 

A2 G h-iTZaP^) 

A3 G hiJZapfl 


Proof. Let x ^ dd{TZa) be the characteristic function of a T-equivariant rank 2 torsion 
free sheaf on Ua with reflexive hull TZa described by toric data (u, v,p). Consider the 
regions (71,(172,Cs,/! of (12). The region x\ci\D has zeros and ones. These zeros form a 


3D partition 'Kf'\ The region x\d has zeros, ones, and twos. The zeros give rise to a 3D 
partition TTjn. The partition TTm has the property that 


are 3D partitions. Finally, consider the connected components of I’s of x\d- Such a 
connected component k has the property that TTin U a is a 3D partition. Moreover, at 
least two of 

U A, U a, U K 

are 3D partitions. If only two of these form a 3D partition, say (without loss of generality) 
U A, U A, then dehne 


( 2 ) 


( 1 ) 


.( 2 ) 




TT^ .— TT^ LJ '^‘2 *— ^2 ^ ^3 

Otherwise choose any two, say and dehne 


( 2 ) 


:= TT. 


( 1 ) 


TT 


( 2 ) 


:= TT 


( 1 ) 


.( 2 ) 




( 2 ) 


U K, 712 := 712 0 K, ^3 T 


db 


Proceeding in this way for all connected components a, we end up with 3D partitions 

Til := Tlj^^ U TTin U U ■ • • , 712 := Tl^^^ U TTin U U ■ ■ ■ ,713 := Tlg^^ U TTin U U ■ ■ ■ . 

Then tt = ( 711 , 712 , 713 ) forms a double box conhguration and the different choices made 
in the construction lead to equivalent double box conhgurations. We described a map 

X{TZa)—> I I n( 7 ?.a, Ai, A2, A3). 

Ai G A{7ZaPi ) 

A2 € KiTZaP^) 

A3 G AiTZapf) 


It is easy to check this is a bijection. 


□ 


Next we assign moduli to each double box conhguration. In the language of charac¬ 
teristic functions: we want to assign a moduli factor to each connected component 
of I’s which is not a local face component. The reason we do not assign moduli to local 
face components is because the rehexive hull TZa is hxed (Remark |2.12[ ). A precise way 
of phrasing this in terms of double box conhgurations is as follows: 


Definition 2.19. To any tt G n(7^„, Ai, A 2 , A 3 ) we associate a space as follows. 
Analogous to Dehnition 2.5, the partition Tiout can be written as a union of connected 
components. Dehne the regions 


Si := {ui -F Ui - 1 } X [U 2 + V 2 , 00] X [U 3 + V 3 , 00], 
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Figure 2. Example with = Ph 

52 := [ui + t>i, cx)] X {u2 + - 1 } X [us + V3, 00], 

53 := [ui + Pi, 00] X [u2 + V2, 00] X {U3 + V3- 1 }. 

Let Pj : —)■ i = 1,2,3 denote projections as before. Let k be a connected 

component of boxes contained in precisely two 3D partitions among tti, 712 , 713 . If 


(13) 


Pi{K) C Pi{TTi n Si 


for all 7 = 1,2, 3, then we associate a copy of P^ to it. (Note that this is independent of 
choice of representative of the equivalence class.) Otherwise the connected component 
of boxes has no moduli assigned to it. We dehne the moduli factor associated to tt as 
the n-fold product of P^, where n is the number of connected components of boxes k 


satisfying (13) 


:= P^ X ■ ■ ■ X P^ 


We illustrate these dehnitions in the following examples. 


Example 2.20. Let tt G n(Pa, (0, 0, 0), (0, 0, 0), (0, 0, 0)) be given by Figure 2. 
The green, blue, yellow 3D partitions 711 , 712,713 are representatives of tt = ( 711 , 712 , 713 ). 
The red boxes correspond to TTin and are contained in all three 3D partitions 711 , 712 , 713 . 
The white boxes correspond to tToui and are contained in precisely two of tti, 712 , 713 . Those 
with moduli have a label m. Note that only one of the three connected components of 
white boxes satishes (13). Therefore Ctt = P^. 


Example 2.21. Let tt G Y\.{TZa: ( 00 , Eb, 0), (0, □, 0), (S- 
ure 3. The dots indicate inhnite legs. One connected component (of vertex type) fails 


, Et)) be given by Fig- 


(13) so has no moduli. The other two connected components (of edge type) satisfy (13) 


so ^ pi X pi. 
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Figure 3. Example with x Ph 


Example 2.22. Let tt G n( 7 ^a, (tizzn, m, 0 ), ( 0 , □, 0 ), m, ttd)) be given by 
Figure 4 (left). Then C,r — However, adding one box in the corner Figure 4 (right) 
kills the moduli and results in C^r = pt. Note that in this example TTin = 0 . 


We learn an important lesson from Example |2.22[ if we are given a double box con¬ 
figuration 77 G n(7?.Q,, Ai, A 2 , A 3 ), then we cannot decide from the Aj alone whether there 
are edge moduli. Informally: the “corners” influence the “legs”. This means we “cannot 
split” vertex and edge contributions. A similar phenomenon occurs when discussing the 
vertex/edge formalism of rank 2 DT theory in Section]^ (see Example 6.13). We are led 
to consider “global” double box conhgurations in the following sense. 


Definition 2.23. Consider a smooth projective toric 3-fold X and a T-equivariant rank 2 
reflexive sheaf 7^ on X dehned by toric data (u, v, p) such that in each chart TZa '■= 'TZ\u^ 
is singular. (The general case, where some TZa can be locally free, is discussed in Remark 
2.26 ) For each edge afd G E{X), £x a double square configuration 

A := {Xal3}al3£E{X)- 


For each a G V{X) we denote its neighbouring vertices by fdi,/d 2 , (^ 3 . The labeling 
is chosen such that Cag^ is dehned by {xj/ = Xi" = 0} for all i = 1,2,3 and i',i" G 
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Figure 4. Example with (left) and C„ = pt (right). 


{1, 2, 3} \ {i} distinct. Fix a double box conhguration 

TT^ G n(7^Q, 7 ^a/33 ) • 

Then the collection tt := {7Ta}aev(x) describes the characteristic function % of a T- 
equivariant rank 2 torsion free sheaf on X with reflexive hull TZ by Proposition 2.18 We 
denote the collection of all such double box conhgurations by n(7^, A). To each tTq, we 
associated a moduli factor (Dehnition |2.19| ). Then 

a£V(X) 

Here x ^ y when for some af3 G E{X), there is a local edge component containing x 
in chart Ua and a local edge component containing y in chart f/^ and both components 
match under gluingj^ We dehne the weight of tt as 

uj{7t) := e{Ci,), 

where e(-) denotes topological Euler characteristic. Recall that for each a(3 G E(X), we 
have (Section |2.1[ ) 

Nccpix — 0^i{mai3) © 

For each ajS G E{X), there are two faces pi^a/s, p 2 ,ai 3 which share a(3 as an edge and two 
disjoint faces ps^a/s, Pi,ay connected by the edge ajS. Suppose face p 3 ,a /3 contains vertex 
a and p 4 ,a /3 contains (3. We dehne the generating function associated to X, IZ, and Xag 

Zx,k,a(9):= E 

^•e^(7^,A) 

a£V(X) al3£E{X) 

3 

fm,m'{X) := ^ ^ {-mki - mk 2 + 1) - ^ (-mfci - mk 2 + 1), 

*=1 {ki,k2)&Xi (fci,fc2)eAout 

gu,u',v,v'W ■= -|A|(m + u' + v + E) + lAsKn + v'), 


^'^Note that it might happen that an edge component with moduli gets matched with an edge com¬ 
ponent without moduli, in which case the relation ~ kills the moduli. 
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for any A = (Ai,A 2 ,A 3 ). There is one issne with this dehnition: /m,m'(A), gu,u',v,v'{^) 
depend on choice of representative of A. In the formula for x(7 T’), it is assumed that 
for all a G V{X), we choose representatives of tTq, with the following 

property: 

For any a[3 G E{X), suppose without loss of generality that Ca /3 is given by {x 2 = = 

0} in chart [/„ and {x 2 = x'^ = 0} in chart Up. Suppose the toric data of TZa in chart 
is given by (mi,M 2 ,W 3 ), (ni,n 2 ,n 3 ) and in chart Up by Then for 

all i = 1, 2, 3 

-Pi(7ra,i \ [ui,ui + iV] X I?) = Piinpp \ [n'l, u\ + A^] x I?) 

A’i(vr „,2 \ [mi, Ml + N]x I?) = Pi(7r^,2 \ K, u\ + A^] x I?) 

Pli.'Xap \ [mi, Ml + A^] X I?) = Pi(7r^,3 \ K, u'^ + A^] X Z^), 

where Pi : —)■ denotes projection as before and A^ 3> 0. 


This says that not only do tTq, and TZp have matching asymptotics (namely Xap)-, but the 
legs of their constituent partitions tTq, j, npp have matching asymptotics too. This choice 
makes it more convenient to do the Cech calculation below. Even though /m,m'(A), 
gu,u',v,v'{X) depend on choice of representatives, the formula for x( 7 r) does not by the 
following proposition: 

Proposition 2.24. Let X be a smooth projective toric 3-fold. Let TZ be a T-equivariant 
rank 2 reflexive sheaf on X described by toric data (u, v,p) and let IZ ^ Q be a T- 
equivariant quotient of dimension < 1. Let tt = {7^a}aev{x) correspond to the charac¬ 
teristic function of the kernel oflZ^ Q, where 

^ n(PQ! , Aq,^j , Aq,^2 5 Aq^j ) . 

Suppose for all a G V{X) we choose representatives of tz a as described in Definition 
\2.k‘A Then 

x{Q) = xi^)- 

Moreover 

C3(S) = 2x(S)- 5^ |A,^|(ci(V)C<,^). 

a/SeEiX) 


Proof. This follows from a Cech calculation using the T-invariant open affine cover Ua, 
a G V{X) like in |MNOPl1, Lem. 5]. The upshot is to choose representatives of each tTq 
as in the previous definition and then calculate the T-representation 

a£V(X) aP&E{X) 

Cancelling infinite terms and finally setting ti = ^2 = ^3 = 1 gives the formula for x{Q)- 
The formula for cflQ) follows from Hirzebruch-Riemann-Roch. □ 


Remark 2.25. In Definition 2.23 we fixed a smooth projective toric 3-fold X. Suppose 
Y is any smooth toric 3-fold with vertices V{Y), edges E{Y), and T-equivariant rank 2 
reflexive sheaf P on Denote by EflY) the edges which correspond to compact lines 


^®When Y is non-compact, one can still associate an o pen polyhedron to it, which is a subset of the 
polyhedron of any toric compactification of X (Definition 4.1). 









24 


A. GHOLAMPOUR, M. KOOL, B. YOUNG 


— Cap C Y. Suppose for each a(3 G Ec(Y), we fix a double square configuration Xap, 
then one can still consider 


7re^(7^,A) 


a&V{Y) 


aP&E,{Y) 


^oc/3 


(Aa/?) + gi 


'P3,a0'^P4,a0^^P3,a0'^P4,ai 


I (Aa/?) ) , 


where ps^ap and pi^ap are dehned as in Dehnition 2.23 In applications, Y has toric 
compactihcation X and TZ is the restriction of a T-equivariant rank 2 reflexive sheaf on 
X. Then Zyu^iq) is part of a generating function on X. 

Sometimes, for computational reasons, it is useful to consider an analog of 
where one hxes a double square conhguration A^^ for all (not necessarily compact!) edges 
a(5 e EiY) 


(14) 


^aeV(i") 




•n-en(7e,A) 

When both dehned, (g) and Zy^^{q) coincide up to an overall multiplicative factor 

of q to some power. This power involves information of the normal bundle to the lines 
Cap^ whereas (g) does not depend on the geometry but only on IZ and A. 

Remark 2.26. It is not hard to redo this section in the case TZa is locally free, i.e. n* = 0 
for some i or pi = pj for some i, j (Proposition [2^. We think of these as degenerate cases. 
Then TZa is T-equivariantly decomposable. In this setting, a double box conhguration is 
dehned as a pair of partitions tt = (tti, 7 ^ 2 ) without any conditions, where we impose an 
equivalence relation on such pairs as before (and similar for double square conhgurations). 
After adapting the formulae for | 7 r|, |A| accordingly, the formula of Proposition 2.24 holds 


for any T-equivariant rank 2 rehexive sheaf on X. The details are straight-forward and 
left to the reader. 

2.6. Combinatorial formulae. Let T = and TZ a T-equivariant rank 2 rehexive 
sheaf on Y with toric data (u,v,p). The following generating function was introduced 
in Remark 12.251 

7ren(7e,A) 


•7 comb 
C^7^,A^ 


Here A = (Ai, A 2 , A 3 ) are three double square conhgurations describing hxed asymptotics 
along the Xi-, X 2 -, and xa-axes. In Section we will see that these generating functions 
form the building blocks for calculating ([^ for rank r = 2 and toric 3-folds X in examples. 
Recall tha t TZ is singular if and only if all Uj > 0 and all pi are mutually distinct 
In the case TZ is locally free. 


2.4). 


(Proposition 

square conhgurations Aj is given by a pair of 2D partitions {Xi, pi) and 


Z™ 3 '^^(g) decomposes. Each double 


(16) 


ycomb 

'^C3,7^,A 


{q)=y\/x,M,xM^, 




(q), 


where denotes the usual topological vertex hrst discovered in |AKMV] ,^ 

This product structure follows from the fact that TZ splits T-equivariantly as a sum of 


definition \N \,^,\2,X3{q) = where the sum is over all 3D pa rtitio ns tt with asymptotics 

Ai, A 2 , A 3 and | 7 r| denotes the renormalized volume mentioned in Definition 2.17 
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two T-equivariant line bundles (Proposition |2.4[ ) , which gives an extra C*-scaling action 
on the factors. Using this extra C*, ( [l^ follows. 

The interesting case is when IZ is singular. The generating functions can 

then be computed by relating double box conhgurations to double dimers. In the case 
of no legs (or small legs) this leads to closed formulae. The following two theorems are 
proved in a forthcoming combinatorial paper |GKY] . The hrst two authors recently found 
a geometric proof motivated by wall-crossing and Hall algebra methods |GK21 IGK3] . 


Theorem 2.27. [GlTyl IGK^ Let TZ be a singular T-equivariant rank 2 reflexive sheaf 
on described by toric data (u, v,p). 


ycomb 

7^,0, 0,0 


Vi V2 vz 

(q) = M{qf nnn 


i=l j=l k=l 


\ _ qi+j+k-l 

X _ qi+j+k-2 ’ 


where M{q) = nA:>o ~ denotes the MacMahon function. 

Theorem 2.28. [GKYl lGK3] Let TZ be a singular T-equivariant rank 2 reflexive sheaf 
on described by toric data (u, v,p). Suppose ( 01 , 02 , 03 ) = (1,1,1) 


(Ai, A 2 , As) = ((0, 0, 0), (□, 0, 0), (0, 0, 0)). 


Then ^ 

z-r(g) = 

Denote by Quotha (JZ, n) the Quot scheme of quotients 

7Z —» Q 


where Q is 0-dimensional and has length n. The following is a corollary of Theorem 2.27 


Corollary 2.29. Let TZ be a T-equivariant rank 2 reflexive sheaf on described by toric 
data {u,\,p). Then 


^e(Quotc3(77,n))g” 

n=0 


M{q)^ 

M{q)^ nrii njLi n 


«3 

k=l 


l — qi+j + k-l 
1 — + — 2 


if TZ is locally free 
if TZ is singular. 


Since Theorems 2.27 and 2.28 play such a central role in our calculations, we give a 


brief outline of the proofs to give a flavour of the ideas involved. This sketch is by no 
means meant to be self-contained. 


Combinatorial proof. We hrst replace our double box conhgurations with diher- 
ent combinatorial objects, namely, conhgurations of the hexagonal-lattice double dimer 
model with certain tripartite boundary conditions, as dehned in |KW 2] . This is essen¬ 
tially done by separating the double-dimer model into two single-dimer models, and then 
using the standard correspondence between the single dimer model on the honeycomb 
lattice and plane partitions, though there is some work to do in order to see that the 
“tripartite” boundary conditions of |KW2] are the correct ones. Figure 5 is the double 
dimer model we associate to the example of the double box conhguration in Figure 2. 

We put these dimer models into a generating function by weighting each with eg™ for 
some c,m E Z>o as we did in the case of double box conhgurations. The correspondence 
between double box conhgurations and double dimer models is many to one, but it is 
weight-preserving, and it does preserve the generating functions. Weight preserving here 

^^Here we take A = (Ai, A 2 , A 3 ) with each A^ = (0,0,0). 
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Figure 5. Double dimer model associated to Fig. 2 


means each single dimer configuration contributes the same power of q as any of the 
double box configurations in its preimage, and that the coefficient of q is the sum of the 
coefficients of q of all double box configurations in its preimage. 

We then compute the generating function of this double dimer model using the tech¬ 
nology of Kenyon-Wilson |KWH IKW2j . It is a certain rational function in partition 
functions of single dimer models on the hexagonal lattice. We realize each of these single 
dimer model partition functions as a product of vertex operators, and use Fock space 
computational techniques due to Okounkov |Oko| lORll IOR2t lORVj to simplify the re¬ 
sult and identify the factor M{qY in the generating function. We then reinterpret the 
resulting quantity as the generating function of a new single-dimer problem - namely the 
classical g-count of boxed plane partitions |Mac] . This last step accounts for the final 


product in Theorem 2.27 


Geometric proof. The geometric proof proceeds as follows. We first realize the left 


hand sides of Theorems 2.27 


Quot schemes of reflexive sheaves TZ as in Corollary 2.29 


2.28 as generating functions of the Euler characteristics of 

Subsequently we apply Hall 


algebra/wall-crossing arguments of |Bril|Joy[IKSllST] in order to to rewrite this as M(g)^ 
times the generating function 


^e(Hilb"(Smg(R),n)),”, 


n=0 


where Sing(77) is a certain 0-dimensional subscheme of supported at the point where 


TZ is singular. This generating function accounts for the final product in Theorem 2.27 


3. Applications to Euler characteristics 

Let X be a smooth projective toric 3-fold with polarization H. Let Al^(2, ci, C 2 , C 3 ) 
be the moduli space of rank 2 /i-stable torsion free sheaves on X with indicated Chern 
classes. Recall that we do not consider strictly p-semistable sheaves. Consider the 
generating function 

Zx,h,ci,c2(?) := X]e(Alf(2,ci,C2,C3))g'=L 

csSZ 
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In the introduction, we noted that the double dual map is a constructible morphism 
(■)** : -Mf ( 2 , Cl, C 2 , C 3 ) □ AT#( 2 , Cl, c', c'). 


C2,C3 


Here (2, ci, C 2 , Cg) is empty unless 

clH 


< C2H < C2H, 


where the hrst is the Bogomolov inequality |HLl Thm. 7.3.1] and the second follows from 
ampleness of H. Moreover, for a given C 2 there exists a constant K{X,H, 01 , 02 ) such 
that A/"^(2, Cl, C 2 , dg) is empty unless jGKlt Prop. 3.6] 

( 16 ) 


0 < C 3 < K{X, H, Cl, C 2 ). 


The generating function Zx,h,ci,c 2 (?) is a Laurent series in Using the methods 
of the previous section, it can be expressed in terms of Euler characteristics of Quot 
schemes as follows: 

• Find a GIT representation of all connected components C of all reflexive hxed loci 
A/]^(2, Cl, C 2 , Cg)^. These are conhguration spaces of points on or open subsets 
thereof. This was shown in |GKlj . E.g. for X = P^, each connected component 
C is isomorphic to either a reduced isolated point pt or C* \ {pt} |GKlj . 

• Compute the Euler characteristics e(C). Euler characteristics of conhguration 
spaces of points on P^ have been computed in general by Klyachko |Kly3| . 

• Let C be a connected component of A/'x (2, ci, C 2 , dg)^. The toric data (u, v, p) of 
any T-equivariant rehexive sheaf [77] G C has hxed u, v and (in general) varying 
p. We stress the fact that u, v are constant over the connected component C by 
writing u(C), v(C). The components of the vectors u(C), v(C) are indexed by 


rays p G F{X) (Dehnition 2.2) 


U(C) = {Up{C)}p^F{xp V(C) = {Vp{C)}p^F{X)- 


The Chern classes 02,03 can be expressed in terms of (u(C), v(C), p(C)) as ex- 

In particular for any a G V{X) either all restrictions TZ\u^ 
C have a singularity at the origin or not. In the former 


plained in Section 
of elements of [77 
case the length o 


2.3 


the singularity is the same for all elements [77] G C and is 
given by the formula of Proposition |2.4[ Since Og is the sum of the lengths of the 
singularities over all charts (Proposition |2.4[ ) , all elements [77] G C have the same 
third Chern class. 

At the level of closed points, the hbre of (■)** over [77] G C is Quot(77, C 2 , Cg)^ 

C2 — C2 + C2, 

Cg = Cg + Cg + ClC^k 


Proposition 


2.11 


implies that all hbres Quotx(77, C 2 , Cg)^ over [77] G C are iso¬ 


morphic and we may write 

Quotx(C, 4, Cg)'^ := Quot(77, 4', Cg)'^ 

for the isomorphism class. The connected components of the hbres Quotj(^(C, C 2 , Cg)^ 
are all products of P^’s and were described in Section 2.5 
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Putting everything together gives the following structure formula 


(17) Z 


X,H,ci,C2 




E 


e(C)e(C')?« 


C3 c^,c^ CCA/'^(2 ,Ci,C 2,C3)^ C'CQuot^f (C,C 2 -C 2 ,C 3 —C3—Ci(c 2 -C 2 ))^ 

In the rest of this section we work out this generating function in the following cases: 


• C 2 is minimal in the sense of Proposition 

• X = and C 2 = 1 , 2 . 

3.1. Minimal second Chern class. We start with two propositions. 

Proposition 3.1. Let X be a smooth projective toric 3-fold with polarization H and fix 
r and Ci. Let C 2 be such that C 2 H is minimal with the property that there exists a rank r 
pi-stable torsion free sheaf on X with Chern classes ci,C 2 . Then for any such sheaf the 
cokernel Q = X**/X is 0-dimensional. 

Proof. Let X be as stated. Since X is /i-stable, its Chern classes satisfy the Bogomolov 
inequality [HLi Thm. 7.3.1]: 

(r - l)c\H 


C 2 H > 


2r 


Since C 2 H is bounded below, it makes sense to speak of C 2 satisfying the property that 
C 2 H is the smallest value for which there exist rank r p-stable torsion free sheaves on X 
with Chern classes ci,C 2 . Suppose Q is 1-dimensional. Use the same notation for Chern 
classes c', c'' as in the beginning of this section. Then > 0 by ampleness, so 

c'aT/ = C2H + c!^H < C2H. 

Therefore X** is a rank r /i-stable torsion free (reflexive) sheaf on X with Chern classes 
Cl, C 2 contradicting minimality. □ 

Proposition 3.2. Let X be a smooth projective toric 3-fold with polarization H and 
fix Cl. Let C 2 be such that C 2 H is minimal with the property that there exists a rank 2, 
pi-stable torsion free sheaf on X with Chern classes ci,C 2 . For each a G V{X), denote 
by Pi,a, P 2 ,a, P 3 ,a ^ the faccs sharing vertex a. Then 


-X,H,ci,C2 


(,) ^ 5 ; e{CW 

c '3 C(ZNfi( 2 ,ci,C 2 ,c'^Y 

''^Pl.a C) ‘^P2,a C) ‘‘’PS^ct O 

X n n n 

a G V{X) 

V[7?.] e C ■ TZ\ Ua is singular 


i=l 


i=l 


n 

k=l 


I _ (^q-‘iy+j+k-i 
X — (^q—‘^y+j+k—2 


Proof. Combining (17), Prop. |3.1[ Prop. |2.24 gives 




UcTllu^, 0 , 0,0 




C'3 C(lNfi{2,Cl,C2,c'^)T 


aeU(X) 


where ^) was introduced in Remark 


is given by Theorem 2.27 


2.25 


The latter generating function 

□ 


If the polyhedron A(X) is “small” (dehned below), the connected components C in 
the previous proposition turn out to be isolated T-£xed points. 
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Definition 3.3. Let A be a convex polyhedron such that each vertex has exactly 3 
edges. We view A up to arbitrary stretching of edges and angles between edges (but not 
contracting them to a point). We say that A is small if the following property holds. 

Condition: For any way of colouring one face red and three faces blue (and not colouring 
any of the other faces), the red face must share an edge with a blue face and at least two 
blue faces must share an edge. 

Example 3.4. Any convex polyhedron such that each vertex has exactly 3 edges and 
< 6 faces is small. There are 4 of these: tetrahedron, cube, prism over a triangle, prism 
over a triangle with one corner cut off. Hence rkPic(X) < 3 ensures A(X) is small. 
Polyhedra with > 7 faces are generally not small. However, a prism over a 5-gon has 7 
faces but is small. Therefore 16 out of 18 smooth toric Fano 3-folds have small A(X) 
(the two with rkPic(X) = 5 do not have small A(X) |Batl IWW] 1. 

Theorem 3.5. Let X be a smooth projective toric 3-fold such that A(X) is small. Let 
H be a polarization and ci a first Chern class such that gcd(2, ciiL^) = 1. Let C 2 be such 
that C 2 H is minimal with the property that there exists a rank 2 jx-stable torsion free sheaf 
on X with Chern classes ci, 02 - For each a G V{X), denote by pi^a, p 2 ,a, P 3 ,a £ F{X) the 
faces sharing vertex a. Then the connected components of fi/ff {2,ci,C2,c'^)'^ are isolated 
reduced points for all c'^. Moreover 


^q-2y+j+k-i 


’'P2,q; 

X n n n n If 

a G V{X) k=l 

TZ\uo, is singular 


In particular, Zx,h,ci,c 2 {q) equals M{q 2 ) 2 e(A) g, Laurent polynomial in q. 


Proof. Let 77 be a T-equivariant rank 2 /i-stable reflexive sheaf on X with Chern classes 
Cl, C 2 , toric data (u,v,p), and characteristic function x- Suppose x h^is a face compo¬ 
nents (Definitions 2.6| 2.7). We claim a = 3. The case a < 3 cannot happen, because 
then 77 is decomposable hence p-unstable. If a > 3, then the connected component C 
of the T-fixed locus containing 77 has positive dimension. We show that this leads to a 
contradiction with A(X) is small and C 2 minimal in the remainder of the proof. Once 
this is established, the formula of the theorem follows from Proposition 3.2 Note that 


for fixed ci,C 2 , the third Chern class Cg is bounded by (16). Since the triple product of 
Theorem 2.27 is a polynomial (counting boxed 3D partitions) polynomiality follows. 


Suppose at a > 3, then the moduli space Nffix) of T-equivariant rank 2 /i-stable 
reflexive sheaves on X with characteristic function has dimension > 0. The space 
of equivariant reflexive sheaves f/ffix) of Remark 2.9 is realized as the GIT quotient 
of a non-empty Zariski open subset Uff C (P^)“ by SL(2,C). We denote its elements 
by (gi ,... ,qa). Let V C (P^)“ denote the Zariski open subset of elements (gi ,... ,qa) 
with all qi mutually distinct. Therefore, there is a Cg such that H/SL(2,C) is a positive 
dimensional Zariski open subset 


H/SL(2,C) C A7f (2,ci, C2, c'^f C Aff (2,ci, C2, c') 
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Consider the classical GIT problem of a (ordered) points {qi ,on We assign 
weights Wi,... ,Wa to these points as follows. The characteristic function x has a face 
components. For each face component, there is a maximal number of faces Pi^,..., pi^. 
such that the corresponding local face components are glued together in x- Iii particular, 
= ... = . To this face component we assign weight 

Wi := {Dp^^H^)vp^^ + ■ ■ ■ + {Dp^^H^)vp,^ > 0 . 


Set w := J2i=i then each Wi is smaller than 

Next we use the assumption A(X) is small. By Lemma 3.6 below, there exists a 
GIT semistable point conhguration (gi,..., g^) such that there are i, j with the following 
properties: 


• Qi = Qj is distinct from all other g^ and all other g^ are mutually distinct, 

• there exists a chart Ua containing the local face components corresponding to 
both i and j. 

Informally: on the polyhedron A(X), there are two adjacent faces whose corresponding 
points on P^ can become equal. The unions of faces 0* of the lemma correspond to 
the unions of local face components glued together by X- Note that Wi + Wj < ^ is 
a strict inequality, because gcd(2, ciH"^) = 1. Therefore, the above point conhguration 
(gi,..., qa) corresponds to a p-stable torsion free sheaf X with characteristic function x 
(and hence Chern classes Ci,C 2 ). The rehexive hull T** is /i-stable with Chern classes 
Cl, C 2 for some d^. Since g^, g^ are equal, the characteristic function of T** is obtained from 
the characteristic function % of X by adding a (non-empty) inhnite leg of I’s. Therefore 
c\y^E > ch 2 iL, or equivalently d<Ji < C 2 Ff, contradicting minimality of C 2 . □ 


Lemma 3.6. Let a > 4, tci, ..., tCa G Zi>o, w := assume 


for all i = 1,... ,a. Let A be a small polyhedron. Let {0* : i = 1,..., a} 6e a collection, 
such that 

• Qi is a union of faces of A, for all i = 1,... ,a. 

• Qi minus the vertices is connected, for all i = 1,... ,a. 

• Qi and Qj have no faces in common, for each i,j = l,...,a,i^j, 

• ur=i0* = A- 

Then there exist Qi, Qj, which share an edge and 

w 

Wi + Wj < —. 

2 


Proof. It suffices to prove the case where each 0j is equal to a single face, 
without loss of generality wi < W 2 < ■ ■ ■ < Wa. Then 


Assume 


w 


Wi+Wi< —, 

for alH = 2,..., a — 1. We hrst prove the lemma for a > 5. Colour the face with label 
Wi red and those with label W2, ... ,tCa-i blue. The result follows from Dehnition 3.3 
For a = 4 we need a slightly more rehned analysis. 


^^This problem is discussed in |Doll Ch. 11]. 
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Case 1: Wi + Then we colour the face with label Wi red and those with label 


W 2 ,Ws,W 4 blue. The result again follows from Dehnition 3.3 


Case 2: Wi + W 4 > Then W 2 + W 3 < ^ 
wi,W 2 ,W 3 blue. The result follows from Dehnition |3.3 


In this case, we colour the faces with label 

□ 


3.2. Applications to projective 3-space. In this section, we calculate Zx,h,ci,c 2 {<i) 
for A = and C 2 = 1, 2. We also classify the rehexive hulls for C 2 = 3. The generating 
function does not depend on H and we drop X and H from the notation. We take 
Cl = — 1 and also suppress it from the notation. Note that rank and degree are coprime 


and there are no strictly /i-semistable sheaves. We use the notation of Example 2.3 


Example 3.7. Let C 2 = 1. Using the methods of Section one can see that C 2 = 1 
is the minimal value for which there exist rank 2 p-stable torsion sheaves on P^ with 
Chern classes Ci, C 2 (see also |GKlj i. There are exactly four such sheaves which are 
T-equivariant and reflexive. Each of them has a length 1 singularity at one of the torus 
hxed points and no further singularities. The toric data (u, v, p) of these reflexive hulls 
is described as follows: 


(i) For any choice i, j, k, I G {1, 2, 3,4} with i, j, k, I mutually distinct 


ui = U2 = U3 = 0 , U4 = - 1 , 

Vi = 0 , Vj = Vk = vi = 1, 

Pjj Vk) Pi mutually distinct. 


Recall that M 2 = W 3 = ^4 = 0 is forced by equation ([^. Note that for hxed u and 
V, different choices of Pj,Pk,Pi give T-equivariantly isomorphic rehexive hulls. 


Theorem 3.5 


gives 


Z,,=i(g)=4(g-i + g)M(g-y. 


We also note that any connected component C of any Alps (2,—1,1, 03 )^ has constant 
rehexive hulls. This is useful for applications to DT theory in Part II. 


Example 3.8. Let C 2 = 2. We describe all possible rehexive hulls of elements of 

□ Alps ( 2 ,- 1 , 2 , 03 ^. 

cs 

Like the previous example, these rehexive hulls are isolated reduced points. However 
this time legs can appear, i.e. C 2 and may diher. There are three types of rehexive 
hulls. Their toric data (u, v, p) is described as follows: 

(i) For any choice i,j, k, I G {1, 2, 3,4} with i,j, k, I mutually distinct 

Ui = U 2 = Us = 0 , U 4 = —2 

Vi = Vj = Vk = l, vi = 2, 

Pi = pj and Pi, pk, Pi mutually distinct. 

These rehexive hulls have C 2 = 2. 

(ii) For any choice i, j, k, I G {1, 2, 3,4} with i,j, k, I mutually distinct 

Ui = U 2 = Us = 0 , U 4 = — 2 , 

Vi = 0, Vj = 1, Vk = vi = 2, 

Pj) Pk) Pi mutually distinct. 

These rehexive hulls have C 2 = 2. 
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(iii) The reflexive hulls of Example 3.7 Recall that these reflexive hulls have C 2 = 1. 
Any T-equivariant rank 2 p-stable torsion free sheaves on with Chern classes 
Cl = — 1, C 2 = 2 has one of the above as its reflexive hull. In cases 1 and 2 the cokernel 
jr** j jr jg 0-dimensional, whereas in case 3 the cokernel is 1-dimensional. Using formula 


(17), Proposition 2.24, and Theorems 2.27, 2.28 one can compute the following formula 


Z,,= 2 (g) = I q\l + + {q-'^ff + g^(l + + 2{q-'^f + (g"^)^ + (g"^)^) 


.- 2\3 


type (i) contribution 


-2 


type (ii) contribution 


(l_g-2)2 (l-g-2)2 

'■-V-' 

type (iii) contribution 

=121 


We also note that any connected component C of any Alps (2,—1, 2, 03 )^ has constant 
reflexive hulls. 


Example 3.9. Let C 2 = 3. We describe the connected components C of 

I_I -Mps (2, —1, C 2 , C^)'^ 

containing reflexive hulls of elements of 

(18) □ Alps (2,-1,3, cs)^. 

C3 

This time, C no longer needs to be isolated. The toric data (u, v, p) of these connected 
components is classified as follows: 

(i) For any choice i, j, k, I G {1, 2, 3, 4} with i,j, k, I mutually distinct 

Ui = U 2 = U 3 = 0 , U 4 = - 2 , 

Vi = Vj = Vk = l, vi = 2 , 

Piy Pj) Pkj Pi mutually distinct. 

These reflexive hulls have C 2 = 3 and form non-isolated connected components 
isomorphic to C* \ pt. See also |GK1] . 

(ii) For any choice i,j, k, I G {1, 2, 3,4} with i,j, k, I mutually distinct 

Ui = U 2 = Us = 0, U 4 = —3, 

Vi = 1, Vj = 2, Vk = 1, vi = 3, 

Pi = pj and Pi, pk, Pi mutually distinct. 

These reflexive hulls have C 2 = 3 and correspond to isolated reduced points. 

(iii) For any choice i,j, k, I G {1, 2, 3, 4} with i,j, k, I mutually distinct 

Ui = U 2 = Us = 0, Ui = —3, 

Vi = 0, Vj = 1, Vk = vi = 3, 

Pj) Pk, Pi mutually distinct. 

These reflexive hulls have C 2 = 3 and correspond to isolated reduced points. 
















RANK 2 SHEAVES ON TORIC 3-FOLDS 


33 


IV 


The reflexive hulls of Example 3.^ 


Recall that these reflexive hulls have C 2 = 1 


or C 2 = 2 and correspond to isolated reduced points. 

Any T-equivariant rank 2 p-stable torsion free sheaf on with Chern classes ci = —1, 
C 2 = 3 has one of the above as its reflexive hull. This time not all connected components 
of ( [I^ have constant reflexive hulls. Connected components with an element having a 
reflexive hull appearing in (i) do not, whereas those having a reflexive hull as in (ii) and 
fiii) do. For fiv') some do whereas others do not. 


Part II: Virtual 


4 . Rank 2 Donaldson-Thomas type invariants 

In this section we define rank 2 DT type invariants on a toric Calabi-Yau 3-fold by T- 
and To-localization, where Tq C T is the torus preserving the Calabi-Yau volume form. 
We start with a discussion on the moduli space and T-equivariant Serre duality. 

4.1. Moduli space. Let Y be a smooth toric Calabi-Yau 3-fold. Recall the notation 


used for toric varieties from Section 2.1 Then Y is non-compact so has no good notion 
of moduli space of stable sheaves on it. We therefore want to compactify Y. 

Definition 4.1. Let Y be a smooth toric Calabi-Yau 3-fold. Let X be a smooth pro¬ 
jective toric 3-fold. We say X is a toric compactification of Y if Y C X is a union of 
some of the invariant affine open subsets {Ua}a&v{x) of X. In this context, we denote 
by V{Y) the collection of vertices a for which Ua C Y. Moreover, we write Ec{Y) for 
the edges spanned by vertices of V(Y). The subscript c indicates that we consider edges 
corresponding to compact lines in Y only. 


Let a G V{Y) and use coordinates {xi,X 2 ,X 3 ) on Ua as in Section 2.1 Define the 
subtorus To C T by = Ip^ Equation ([^ implies that there exists a Tg-equivariant 
isomorphism 

Kx\y = Oy. 

Let a(3 G EciY). Then the degree of Nc^^/y is —2 by the Calabi-Yau property, i.e. 

map + m’ap = - 2 , 

where map-, m'^^ were defined in Section 


2.1 


Example 4.2. Important examples of toric compactifications are 

c P^ 

Tot((!lpi(fc) © dpi(—2 — A;)) C P(dpi(/c) © dpi(—2 — fc) © dpi), 
Tot(X5)cP(X5©d5), 

where Tot denotes the total space of the indicated vector bundle, /c G Z, and S is any 
smooth projective toric surface. These compactifications satisfy H^{KV) 7 ^ 0. 


22i 


The torus Tq preserves the Calabi-Yau volume form on Y. 
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Let X be a smooth projective 3-fold over C with polarization H. We denote by 
A4x ■= A4^(r, Cl, C 2 , C 3 ) the moduli space of /i-stable torsion free sheaves on X of rank r 
and Chern classes ci, C2, C3. As mentioned in the introduction, we do not consider strictly 
p-semistable sheaves but work directly on the open subset of /x-stable sheaves. Suppose 
A is a toric compactihcation of a toric Calabi-Yau 3-fold Y (Definition |4.1[ ). We dehne 
a moduli space of /r-stable sheaves on Y using the compactihcation X as follows 

M Ycx := C2, C3) := {[X] G Mx \ X**/7 has support in Y}. 

Proposition 4.3. Ci, C 2 , C 3 ) C 01 , 02 , 03 ) is open and T-invariant. 

Proof. We hrst prove Mycx is T-invariant. Let [X] G Mvcx and t G T be closed 
points. Then t induces an automorphism t : X ^ X. By dehnition, X hts into the short 
exact sequence 

0 — yX^X**^Q —^ 0, 

where Supp(Q) C Y . This sequence remains exact after applying t* . Since t* commutes 
with taking (double) dual, we have 

{t*X)**/t*X = t*Q and Supp(rQ) = t(Supp(Q)). 

Since Supp(Q) C Y and Y is T-invariant, the result follows. 

Next, we prove Mycx is open in Mx by proving that its complement is closed. 
Suppose (7 is a smooth quasi-projective curve and let 0 be a point on C . Suppose we 
have a morphism C —>■ Mx such C \ 0 maps to Mx \ Mycx- It suffices to show 
0 also maps to Mx \ Mycx- Let X be the family induced by this map. We know 
Supp((J-'i)**/J-t) intersects X \ Y for all closed points t E C \ 0 and we want to show 
Supp((J-o)**/Xo) intersects X \ Y. 

After possibly removing a hnite number of points from C'\ 0 , the rehexive hulls {Xt)**, 
t E C \ 0 form a hat family |Kolj . We can take the hat limit of this family inside the 
(proper) moduli space of Gieseker semistable torsion free sheaves containing {Xt)** with 
t G G \ 0 to hll in the missing hber over 0. This gives a hat family TZ over C such that 
TZt = (Xt)** for all closed points t E C \ 0, but TZq may not be equal to (To)** (because 
rehexive hulls of members of a hat family may jump). In particular, TZq may not be 
rehexive. A priori, TZq is only Gieseker semistable and hence /i-semistable. 

Over G \ 0 we also have the hat family of quotients given by the short exact sequence 

(19) 0 ^ Xt^Xt—^ Qt^O. 


We take the hat limit of these quotients inside the corresponding relative Quot scheme 

(20) Quotj5fxc/c("^)-L*) ^ 

where P is the Hilbert polynomial of Xq with respect to a polarization on X. The hat 


limit exists, because the map (20) is proper. We obtain a short exact sequence 

■X' 


0 


X 


Q —y 0 


extending (19). We claim that Xq = Xq. Since X'q and Xq only diher in codimension > 2, 


/iSS 

X 


Xq is /i-semistable. There exists a projective moduli scheme of /i-semistable sheaves M 
constructed by D. Greb and M. Toma |GT] . Seperatedness of Mff^, implies equality of 
the equivalence classes of To, Tq in Mff^. This translates into 
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where gr^{-) denotes the graded object associated to a choice of /i-Jordan-Holder hltra- 
tion and [■]** denotes double dual |GTl Thm. 5.5] Since is /i-stable this equality 
implies cannot be strictly /r-semistable. Since is /i-stable, separateness of M.x 
implies = 

Although TZq and (-Fo)** may not be equal, we claim that (T^o)** — We 

proved injects into TZq with cokernel Qo, so they are isomorphic on the complement 
of Supp(Qo)- Since Supp(Qo) has codimension > 2, their reflexive hulls are completely 
determined on X \ Supp(Qo) and (T^o)** — |Har2| Prop. 1.6]. From the chain of 

the inclusions Xq —)■ T^o we deduce 

Supp(7^o/Xo) C Supp((Xo)*7-^o). 

So if Supp(7^o/-7)) intersects X \ F, we are done. For this, it suffices to show that the 
locus of quotients in Quotxxc/ci'^y P) whose support intersect X \ Y forms a closed 
subset. 

Let Q be any S'-flat family of torsion free sheaves on X x S' for arbitrary base S. 
Suppose the Hilbert polynomial of the members of the family is P with respect to some 
polarization on X. Consider the corresponding relative Quot scheme 

Q := Qnotxxs/si^^P)- 

Let Q be the universal quotient over X x Q. Then both Supp(Q) and (X \Y) x Q are 
closed subsets of X x Q and so is their intersection Z. Since X is proper, the image of 
Z under projection to Q gives a closed subset. □ 

In the case r = 1, the moduli space Xl^(l, ci, C2, C3) is independent of the polarization 
H. Moreover, it is isomorphic to a Hilbert scheme I^{X,f3) of ideal sheaves Ic C Ox 
of closed subschemes C of dimension < 1 with [C] = (3 and xi^^c) = X- Here [3 is 
Poincare dual to C 2 and y is determined by ci, C 2 , C 3 . In this case, Mvcx is the open 
subset of closed subschemes C with support in Y. From this description, it is clear 
that Aiycx is independent of the choice of toric compactihcation X. There is no such 
independence in the case r > 1. In general, not only does Xiycx depend on the choice 
of toric compactihcation X, it also depends on the choice of polarization iL on Xj^ We 
think of Xiycx as a moduli space of rank r /i-stable torsion free sheaves on Y. 


4.2. Serre duality. On Xiycx, we have a type of “Serre duality in the X-group” as 
we will see in Proposition 4.6 below. Later we dehne DT type invariants of Y by virtual 
localization on Mycx- Roughly speaking. Proposition]^ states that at the X-group 
level, DT theory on A4ycx is symmetric. This will allow us to specialize the equivariant 
parameters si -|- S 2 -l- S 3 = 0 in Proposition-Dehnition 4.11 We start with some notation 
and a technical lemma. 


Definition 4.4. Let X be a smooth projective variety, F a X-hat family of coherent 
sheaves on X over a base scheme B, and £ a line bundle on X x B. We use the common 
notation 


RjYomp_g(F, F 0 £) := RpB*P0^om(F, F ® £). 


^^For any /x-semistable sheaf Y and choice of /z-Jordan-H61der filtration of J-, the graded object 
gr^{tF) depends on the choice of filtration. However its double dual is independent of this 

choice (up to isomorphism) |HL1 Cor. 1.6.10]. 

^■^In fact, for certain choices of H the moduli space Mx can be empty. For examples, see [GKlj . 
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Moreover, we denote the corresponding it'-group class by 

(F, F 0 £) := [RM’onipgi^,¥ 0 £)] G Kq{B). 

The corresponding trace free parts are denoted by RJ^ompg(¥,¥ 0 £)o, (F,F 0 C)q. 
In the it'-group, we often omit square brackets [■] for brevity. Note that in the case 
F = p*xR and £ = p*xL, we have 

{Px^ 5 Px^ ® Px^)o — {R'iR 0 T)o 

On Kq{B), we have an involution (•), induced by taking the derived dual (■)^. If X is 
toric and F is T-equivariant, then (F, F 0 £) is an element of the T-equivariant X-group 
Kq{B) where we endow B with the trivial T-action. Since (■)^^ = (•), the operation (■) 
is an involution on Kq{B). For any coherent sheaf R on B, we have 

R 0 0 e Ro(R)- 

Lemma 4.5. Let X be a smooth projective variety with polarization H and let Mx = 
j\4x(r, Cl,... ,Cn) be the moduli space of p-stable torsion free sheaves on X with indicated 
Chern classes. Let ¥ be a B-flat family of torsion free sheaves in Mx over a variety B, 
such that all members F;,, b G Bd have the same reflexive hull TZ. Then there exists a 
line bundle L on B and a short exact sequence 

0 —^F — yRML ——)- 0 , 

where Q is B-fiat. For each b G B^, this induces a short exact sequence 

0 —)■ Ffo —> TZ —> Qb —> 0, 

where the first map is the natural inclusion F;, (¥b)** =1Z. 

Proof. For F any X-flat family, Kollar |Kol] constructs a stratihcation Si of B such that 
over each stratum Bi, F|xxBi has a Xj-flat hull 

IFUxSi —t Gj. 

Here G* is a Hj-flat quasi-coherent sheaf and the morphism restricts to the natural 
inclusion Ff, (F^)** for all b G B^. By the proof of [Kol, Thm. 21], the stratification 
only depends on the Hilbert polynomials of (F^)** for all b G B^. In our case, all members 
of F have isomorphic reflexive hulls, so Hollar’s stratihcation is trivial and we denote the 
hull by 0 : F —)■ G. Then G is hat over B by the triviality of the stratihcation and 
Ffc —)■ Gb = (Fb)** is the natural inclusion into the rehexive hull for all b G B^. We 
claim 0 : F —)■ G is injective with H-hat cokernel Q. Consider the exact sequence dehned 
by taking kernel and cokernel 



Fibres over b G B^i give short exact sequences 

0 ^ Fb (Fb)" ^ Qb ^ 0, 

where the hrst map is the natural inclusion. Since F is H-hat and (Fb)** = TZ hj 
assumption, we conclude that the Hilbert polynomials of Qb are constant. Hence Q is B- 
hat. This implies IK is H-hat. Therefore Kb = 0 for all closed points 6 G H, so p_b*K = 0 
and K = 0. 
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Finally, we show G = IZM L. Since is /x-stable, its reflexive hull TZ is /r-stable. 
Therefore, TZ is an element of a moduli space of /i-stable reflexive sheaves Mx on X. The 
S-flat family G gives a morphism 

Suppose Nx has a universal family M. Then 

G = (/ X 

for some line bundle L on B. Since / factors through the closed point b = {[7^]} the 
result follows from M|fc = TZ. In general, N'x may not have a universal family. In this 
case, an etale cover of N'x does have a universal family |HLi 4.D.VI]. Working on the 
etale cover, the same result can be obtained. □ 


Proposition 4.6. Let Y be a smooth tone Calabi-Yau 3-fold and X be a tone com- 
pactification of Y. Let H be a polarization on X and Aiycx ■= oi, C2, C3), 

Aix ■= N^(r, Cl, C2, C3). Let ¥ be a B-flat family of torsion free sheaves in Mx over 
a variety B. Assume all members F;,, b G Bd lie in Mycx and have the same reflexive 
hull TZ. Then 


{TZ, 7^)o G Os - (F, F)o = -((7^, 7^)o ® Os - (F, F)o) G Ko{B). 

If ¥ is a Tfi-equivariant family, then the same equality holds in Kq°{B). If ¥ is a T- 
equivariant family, then 


{IZ, IZ)q ® Ob — (F, F)o — —{{TZ, IZ)q ® Ob — (F, F)o) ® ^ £ Kq {B). 

Proof. Consider the short exact sequence of Lemma 4.5| 

( 21 ) 0 —^F— >IZ^L ——^ 0 . 

From this short exact sequence, we get exact triangles 


(22) RjYompg (F, F) — > RJifompg (F, TZ^ L) — > RjYompg (F, Q), 

(23) RJY’ompg{Q,IZ Kl L) —)■ i?Hom(7^,7^) (g) Os — > RJYompg{¥,IZ Kl L). 
The natural map 

Ox —^ tXfom{IZ, Rf) 

induces 

RT{Ox) —^ i^Hom(7^,7^) ^ i^Hom(7^, 7^)o. 

We define 

A‘ := Cone{Rr(Ox) ® Os —>■ RJfbmpg(¥, IZ Kl L)}, 

where the map is the natural composition 


RV{Ox) ®Ob^ i^Hom(7^,7^) (g Os — 
This leads to the following equalities in the iC-group 
(F,F)o = [kl-]-(F,Q) 

= {IZ, IZ)ii 0 Os — (Q, IZ [ 


RjYompg {¥,IZM L). 


\L)-{¥,Q). 


The first equality follows from (22) and the definition of A". The second equality follows 
from (23) and the definition of A‘. Repeating the same reasoning on (21) tensored by 
p*xKx gives 

(F, F 0 p\Kx)o ={IZ, IZ 0 Kx)o 0 Os — (Q, IZ^ LM Kx) — (F, Q 0 p*^Kx) 

= {IZ, IZ 0 Kx)o 0 Os — (Q, 7?. Kl L) 0 tMt^ — (F, Q) 0 tit2t3. 
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where we used that Q is supported in y x B C X x B. We deduce 

(F, F)o - (F, F ® p*xKx)o ® 

= {7Z, 7V)q ® Ob — {B., 7Z ® Kx)o 8 > Ob ® 

(Equivariant) Serre duality gives the results. □ 


4.3. Donaldson-Thomas invariants. When X is a smooth projective 3-fold with 
7 ^ 0 (e.g. Calabi-Yau or Fano), the moduli space M.x carries a natural perfect 
obstruction theory constructed in |Thoj 

(24) E' —L_a4 ^, 

where E‘ is a 2-term complex of locally free sheaves on Aix and is the truncated 
cotangent complex of A4x- Over a closed point [E] G M-x-, we have 

^i?Hom(X,X)o[l]. 

If X is Calabi-Yau, then the virtual dimension vd of this perfect obstruction theory is 0. 
If X is toric, the above perfect obstruction theory is T-equivariant. A smooth projective 
3-fold X cannot be Calabi-Yau and toric at the same time. 

\i M.X is compact, then the perfect obstruction theory E‘ gives rise to a virtual cycle 
[Adx]™- In the case X is Calabi-Yau, the Donaldson-Thomas invariants are dehned as 

deg([Wlx]"'') e Z. 

For other 3-folds, we can take any a G H*{Aix, Q) of degree vd and consider 


a G Z. 


'[M, 


When X is toric, a can be chosen in H^{Xix, Q) and the invariant lies in 

= Q[si,S2,S3]. 

In this case, one can use the virtual torus localization formula of T. Graber and R. Pand- 
haripande to compute the invariant as explained in EE]. This works as follows. The 
T-£xed part of E’ induces a perfect obstruction theory on Moreover, the virtual 

normal bundle is dehned as 

where (■)™' means taking the moving part EE]. The complex N™ = {Wo —)■ IFi} is a 
T-equivariant 2-term perfect complex. Consider the insertion 




Q(si, § 2 , S 3 ), 


where e(-) = c^p(-) denotes top T-equivariant Chern class. The virtual localization 
formula states 


a = 






e(Y™) 




The rank 1 DT theory and its connections to Gromov-Witten theory have been studied 
in the seminal papers |MNOPl| IMNOP2] . 
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4.4. Rank 2 DT type invariants by T-localization. Let X be a smooth projective 


toric 3-fold with ^ 0. Consider the moduli space M.x- So far in this section, 

the rank r was arbitrary. From now on, r = 2. In Section we studied AiJc for r = 2 
and showed it is smooth (Theorem |2.10[ ). If Aix is projective, then 

1 


[M 


x\ - 


(25) 


E 

CcM'^ 

E 

CcM^ 


n[C]' 


e{N™) 
e{Tc - n [C] 


where the sum is over all connected components C C i C <Z Mx denotes inclusion, 
and Tc denotes the tangent bundle of C. The hrst equality is the virtual localization 
formula and the second equality follows from smoothness of C as in |PT2l Sect. 4.2]. In 
Section]^ we develop a vertex/edge formalism for this class. In what follows, we do not 
assume Mx is projective. 

Suppose y is a smooth toric Calabi-Yau 3-fold wit h toric compactihcation X with 
7 ^ 0. Openness of A4ycx (Proposition |4.3[ ) implies that it inherits the DT 
perfect obstruction theory from A4x- Unlike the rank 1 case, for each closed point 
[X] G A4ycx, the class 

^•^I[.F]=-(A,A)o 

does not have rank zero even though Supp(A**/A) C V. However, the class 

has rank zero and a convenient symmetry by Proposition |4.6 We therefore want to 
“split off” the part —{TZ,TZ)q. In order to be able to do this in families, we make the 
following assumption: 


Assumption 4.7. Let C C be a connected component such that C C Mycx- 
Assume all closed points of C have the same (i.e. isomorphic) reflexive hull 7Z. 


Remark 4.8. By Remark 2.9 Assumption |4.7| implies 7Z is an isolated reduced point in 
the T-£xed locus (i.e. Ext^(7?., 7?.)^ = 0) and C is a product of P^’s. Conversely, suppose 
7^ is a T-equivariant rank 2 /r-stable reflexive sheaf on X with Ext^(7?., 7?.)^ = 0. Any 
connected component C C M.'ycx with all closed points of C having reflexive hull TZ 
is isomorphic to a product of P^’s, hence compact. This fact follows from the toric 


description of Section |2.4[ 


Example 4.9. Let X a smooth projective toric 3-fold with rkPic(X) < 3. Let H be 
a polarization on X and Ci,C 2 Chern classes such that gcd(2,ci7L^) = 1 and C 2 H is 
minimal such that there exist rank 2 /r-stable torsion free sheaves on X with Chern 
classes ci,C 2 . Then for any connected component C of M."x-, all closed points of C have 


the same reflexive hull (Theorem 3.5). In this case the cokernels X**/X of closed points 


[X] G C are always 0-dimensionaI by Proposition 3.1 E.g. Example 3.7 


Example 4.10. In Example 3.8 the closed points of C have the same reflexive hull for 
all connected components C of However in this case X**/X can be 1-dimensional. 
In Example |3.9 some components have constant reflexive hulls whereas others do not. 


Let C C Al3f be a connected component satisfying Assumption 4.7 


Recall that 

Hom(7^, 7^)0 = 0 and Ext^(7^, 7^) = 0 since ^ 0. Since by assumption 
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Ext^(7^, 7^)^ = 0 , the following expression is well-defined 

e(-(7^, 7^)o ® Oc) 

Therefore we can factor it out 

^ n [C]"" = e(Tc - E--') n |C] 

= e((7^,7^)o ^ Oc) ■ e{Tc - E''^ - (7^,7^)o ® Oc) n [C]. 
We therefore write somewhat sloppily 

^ n [C]"-- = e{Tc - - (7^, 7^)o 0 Oc) n [C]. 


e(iVvir) e((7^,7^)o0Oc) 

Here + {TZ, 1Z)q ® Oc has rank zero by Proposition |4 . 6 


Proposition-Definition 4.11. Let Y be a smooth tone Calabi-Yau 3-fold with tone 
compactification X with ^ 0 and polarization H. Let Mx ■= Alx(2, Ci, C 2 , C 3 ), 

M YCX ■ — M^^x (2 ,ci,C2 ,C3) and let C C be a connected component satisfying 

Assumption \4.3\ with reflexive hull TZ. Define 

1 



The specialization 


is well-defined. 


DT(C) 


Sl+S2+S3=0 


e Q 


Proof. As mentioned above, Assumption |4 . 7| implies C = (P^)'^ for some A^ > 0 (Remark 
2.9). By Remark 2.9 there exists a universal family F over C which hts in a short exact 


sequence 

0 —^ F —^ p*xn —^ Q —^ 0, 

where the cokernel Q is a C-flat coherent sheaf and its class in (C) can be expressed in 
terms of the classes := ci(C>pi(l)) of the factors of C = (P^)^ and classes of structure 
sheaves of toric lines and hxed points in X tensored by equivariant line bundles. Using 
this, we show in Section (Theorem 5.2) that 

E-^ Yin,'}Z)o®OceK^{C) 

is a hnite sum of elements ctf^Tfl^tf^ ^ where each coefficient c is the class of a line bundle 
(possibly trivial) on a factor P^ of C = (P^)-^. 

Consider E''^ -|- (7^, TZ)q®Oc. We hrst consider terms in this expression which are not 
To-£xed. By Proposition 4ffi such terms come in pairs 

„J.Wl,W2.LWi _ 

I2 ''3 ''1 ''2 ''3 1 

where c G Pic(P^) for some factor P^ of C = (P^)^. Each such pair contributes 
ci(c*) -b {-wi - l)si -b (-W 2 - l)s 2 + (-W 3 - 1 )S 3 


Ci(c) -b tClSi + W 2 S 2 + tC3S3 


= - 1 , 


Sl-|-S2+S3=0 


where we use ci(c*) = —c. 

Next we consider To-fixed terms of -b (7^, 1Z)q ® Oc, which (a priori) could lead to 
poles when specializing. Each To-£xed term has a weight of the form (tit 2 t 3 )"'. We hrst 
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consider all terms with w G {0, —1}. The T-hxed part of E''^ is the tangent bnndle Tc 
and its dnal 

-Tc ® 

has weight w = —1. Denote all T-£xed terms not appearing in Tc by —a^, where 
Ofc G Pic(P^) for some factor of C = (P^)^ and k rnns throngh some (possibly empty) 
index set. Note that these terms mnst come from obstructions and hence have a minus 
sign. Each such term comes with a dual term a\ ® by Proposition 4.6 In 


conclusion, the contribution of all terms with weight w G {0, —1} to the integral (26) is 

N , s N 

Cl (C-fc j 


(27) 


- s) X n r" 


n(-2ft. - s) X n 


Cl {Ojk) 


i=i k ^ i=i k - 

where s := Si + S 2 + S 3 . 

Finally consider a To-£xed term of + {TZ, TZ)o ® Oc with w ^ {0, —1}. Denote the 
terms occurring with a plus sign by 

hi{tit2hY\ 

where bi G Pic(P^) for some factor P^ of C = (P^)-^ and I runs through some (possibly 
empty) index set. By Proposition 4.6, such a term comes with a dual term 

\-Wl-l 


-b*i{tit2t3 


Together their contribution to 
ci{b^) - {wi + l)s 


IS 


n 


(28) 


ci{bi) + wis 


n[ 

I 

n[ 


{wi + l)s 


WiS 

{wi + 1 ) 

Wi 


+ - 


WiS 


+ 


{wi + l)s 
{wisY 


Ci{bi) 


+ 




-ci{b{) 


Multiplying out (27) and (28) gives the contribution of all To-£xed terms. We see that 
no powers of 1 /s^^^can occur since C = (P^)^ and each factor 1/s comes with a class of 
degree 1. Moreover after multiplying out (27) and (28), each non-zero term containing 
a factor Ci(afc)/s with G Pic(P^) and iFequal to the ith factor of C also contains 
—2hi — s, which cancels the pole. Similarly for ci( 6 /)/s. Here we use that the factors of 
C are 1-dimensional. We conclude that all poles cancel. □ 


4.5. Rank 2 DT type invariants by To-localization. As in the previous sections, 
we consider a smooth toric Calabi-Yau 3-fold Y with toric compactihcation X with 
7 ^ 0 and polarization H. Let Mycx C Mx be as before. Consider the 
subtorus To C T dehned by tit 2 t 3 = 1. In |PT21 Conj. 2], Pandharipande-Thomas 
conjecture that the To-fixed locus of the moduli space of stable pairs is smooth. We 
conjecture the analog in our setting (see Remark 6.23 for some discussion): 


Conjecture 4.12. Let X be a smooth projective toric 3-fold with polarization H and let 
Mx ■■= M^{2 ,cyC 2,C3). ThenM^f is smooth. 


We consider connected components C C M'^ such that C C Mycx- We want to 
endow C with a symmetric perfect obstruction theory coming from To-localization. In 
the stable pairs setting of |PT2j this part is automatic. However, in our case one must 
factor out the part coming from reflexive hulls similar to the previous section. In order 
to achieve this, we make an assumption: 
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Assumption 4.13. Let C C be a compact connected component such that C C 
Mvcx- Assume all closed points of C have the same (i.e. isomorphic) reflexive hull 7Z. 
Moreover, assume Ext^ (71,71)^° = Ext^ (7Z, 71)'^° = 0. 


Remark 4.14. Compared to the T-equivariant case (Assumption 4.7), there are two 
more assumptions: Ext^(77,77)^° = 0, and Ext^(77,77)^° = 0. In the T-equivariant case, 
the hrst vanishing was a consequence and the second vanishing was not needed. We have 
evidence (from examples) that in this case both vanishings are consequences as well. 

Then C has an induced perfect obstruction 


Let C C M'^ satisfy Assumption 


4.13 


theory from To-localization. Assuming Conjecture 4.12 we obtain 
^ n [C]''"’’ = e(Tc 


e{N 


vir,0') 


T*^) n [C], 

= e((77, 7Z)o 0 Oc) ■ e(Tc 


-{7Z,7Z)o^Oc), 


where 


N' 


vir,0 


_ p»V|mo 

~ IC 5 


denotes the virtual normal bundle and tuq denotes the To-moving part. 

Definition 4.15. Let T be a smooth toric Calabi-Yau 3-fold with to ric co mpactihca- 
tion X with ^ 0 and polarization 77. Suppose Conjecture 

Mx := Alf (2 ,Ci,C2 ,C3 ), AIycx := Aly^ 2 , Ci, C2, C3), and let C C 


4.12 




holds. Let 
be a con¬ 


nected component satisfying Assumption 4.13| with reflexive hull 77. We dehne 

^ e(AvinO) e((77,77)o0Oc)’ 


where e(-' 

Oi My 


J[CY 

denotes the Tg-equivariant Euler class. Rec all th at C is a connected component 
and not My^x Proposition-Dehnition ■ 


4.11 


Therefore, we allow ourselves 
to use the same notation. A priori, these invariants have equivariant parameters, but 
Proposition |4.16 below implies that they are numbers. 


The To-equivariant DT invariants of Dehnition 4.15 and the T-equivariant DT invari¬ 


ants of Proposition-Dehnition 4.11 are related in the obvious way. 


Proposition 4.16. Let Y be a smooth toric Calabi-Yau 3-fold with toric corrwactification 
X satisfying 77'^{Kf}) 7 ^ 0 and polarization 77. Suppose Conjecture 4.12 holds. Let 


Mx ■= Mx( 2 , Cl, C2, C3), Mycx ■= d^Ycxi^j *^1! ^2, C3), and let C C M jf be a connected 


component satisfying AssumptionlfC^ with reflexive hull 77. Then T/Tq = C* 
and the connected components Ci of 


are connected components of M'^ 


Y<zx- 


acts on C 
Moreover 


DT(C) = DT(Ci 

CiCCC* 


Sl+S2+S3=0 


Proof. The formula follows by using the T/Tq = C*-action on C and applying virtual 
localization to DT(C). □ 

Theorem 4.17. Let Y be a smooth toric Calabi-Yau 3-fold wi th tor ic compactification 
X with 77^{Kf}) 7 ^ 0 and polarization 77. Suppose Conjecture 4-12 holds. Let Mx ■ = 
Al;^(2, Cl, C2, C3), let Mycx '■= Aly(-x(2, ci, C2, C3), and let C C M'x be a connected 
component satisfying Assumption 4C3\ with reflexive hull 77. Then DT theory induces a 
symmetric perfect obstruction theory on C. Moreover 


+ ( 77 , 77)0 0 Oc = -{E-^ + ( 77 , 77)0 ® Oc) e K^flC). 
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Proof. Let F be the universal family on X x eg Then F has a To-equivariant structure. 
By Assumption 4.13 and Lemma 4.5, we have a short exact sequence 

(29) 0—^F— >n^L ——^0. 


The argument starts as in Proposition 4T From the above short exact sequence, we 
obtain exact triangles 

> RJ^OTUp^iW, TZ Kl L)^° - 

> (i^Hom(7^,7^) (8)C>c)^° 


(30) RJiforrip^ (F, F)^° 

(31) RJ^omp^(Q, TZ Kl L)’^° 
The natural map 


RJiforUp^ (F, Q)'^°, 

-)■ RJ^orrip^ (F, TZ Kl L)^°. 


Ox —t tTifouiiTZy TP) 
i^Hom(7^,7^)^° —^ i^Hom(7^,7^)o°. 


induces an exact triangle 

We dehne 

A‘ := Cone{(Rr(Ox) 0 Oc)^° ^ RJ^omp^(F, TZ ^ L)^°}, 

where the map is the composition 

(Rr(Ox) ® Oc)^° —^ (i^Hom(7^,7^) (8 Oc)^° RJf’onip^ (F, TZ ^ L)^°. 


From (31) and the dehnition of A’, we obtain the exact triangle 

(i?Hom(7^, 7^)o 0 (Pc)^° —> A‘ —> RJ^omp^{Q,TZM L)'^°[l]. 

By Assumption 4.13, i?Hom(7^, 7 ^)q° = 0, hence 

A‘ ^ RJPbmp^(Q,TZ^L)^°ll]. 

Combining this isomorphism with ([SOj), gives an exact triangle 


(32) RJP’omp^ (F, F)g° —> RJ^orrip^ (Q, TZ Kl L)^° [1] —> RJ^orUp^ (F, Q)"^”. 

We can apply a similar reasoning to ( [2^ tensored by p^Kx- This gives an exact triangle 
RJifomp^{F,F<^p*xKx)o° —)■ RJ^omp^{Q,TZMLMKx)'^°[l\ — > RJ^omp^.{F,Q®pxKx)'^°, 
where we used 

i?Hom(7^,7^(8)Xx)o° = /^Hom(7^, 7^)o°''[-3] = 0. 

Since Q is scheme-theoretically supported onX xCcXxC and Y is Calabi-Yau, we 
have 

Q®p*xKx = Q. 

We obtain the exact triangle 

(33) RJYonip^ (F, F ® p*xKx)o° —)■ RJY’omp^ (Q, TZ Kl L)'^° [1] —)■ RjYomp^ (F, Q)^°. 

Combining exact triangles (32), (33), and a non-zero section Ox ^ TCf^ gives an iso¬ 
morphism 

RjYomp^iF ,F)l° = RJ^omp^iF ,F ® p*xKx)l° = RM’omp^iF [-?,]. 


^^Using our assumptions on TZ and smoothness of C, one can show that C is scheme-theoretically 
isomorphic to a connected component of the To-fixed locus of the Quot scheme Quotjj^ (7^) of quotients 
TZ^ Q. The universal sheaf F is the kernel of the universal quotient p*x'P —t Q restricted to X x C. 
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This isomorphism provides the non-degenerate symmetric bilinear form required for a 
symmetric perfect obstruction theorj]^ 

E-\l- = (B-|?)“ll]. 


Next, the second part of the theorem. Exactly as in the proof of Proposition |4.6 
obtain the following equalities in Kq°(C) 


we 


(F, F)o —(7^, 7^)o ® Oc — (Q, L) — (F, Q), 

(F,F ®p\Kx)o ={n,n® Kx)o®Oc - (Q,7^KLKiPx) - (F, Q 0 
= (7^, TZ ® Kx)o ® Oc — (Q, 7Z^ L) — (F, Q). 

Subtracting these equations and using To-equivariant Serre duality gives the desired 
result. □ 


5. Vertex/edge formalism 

Let X be a smooth projective toric 3-fold with ^ 0. Let iL be a polarization 

on X and let 

Mx ■■= (2,Ci,C2,C3). 

Let be a connected component of the fixed locus where x ^ '^{ 2 ^c %2 cs) 

(Theorem 2.10). Recall that the space Xixix) is smooth and equal to the following GIT 
quotient (Theorem 2.8) 

^x(x) = USix) X (P')” X (P‘)' / SL(2.C), US(x) C (P‘)“. 

We will write C = Xixix)- fhis section, we assume C satisfies: 

All closed points of C have the same (i.e. isomorphic) reflexive hull IZ. 


Recall that there are plenty of examples of such C (Examples |4.9| and 4.10). In applica¬ 
tions, X is often a toric compactihcation of a toric Calabi-Yau 3-fold Y and C C M-y^x 


as in Section 4A Assuming C has constant reflexive hulls has several nice consequences: 

• C has a universal family F (Remark |2.9[ ). 

• We have a short exact sequence 

0 —^ F —^ p*xTZ —^ Q —^ 0, 

where the cokernel Q is C-flat and Ext^(7^, 7^)^ = 0 (Remark |2.9[ ). 


• C is a co nnec ted component of the hxed locus of the Quot scheme Quotj('(7^, c'), c'fl)^ 
(Section |2.5[ ). Therefore, C is isomorphic to a product of P^’s. 

Remark 5.1. In this section we set up a vertex/edge formalism closely following |MNOPl] 
in the rank 1 case and |PT2j in the stable pair case. Since our hxed loci are non-isolated 
(products of P^’s) our situation is most similar to [PT2j . In order to follow their method, 
we need the existence of a universal family on C, which is ensured by assuming C has 
constant rehexive hulls. Unlike the case of |PT2] . we can have edge moduli, i.e. moduli 
in the legs (Example |2.21 ). 

^®For a symmetric perfect obstruction theory, it is also required that [1] = 9, where 9 : E’ ^ Lm is 
the map to the cotangent complex [Beh] . In our applications, we only use the isomorphism = E’ 

(namely for calculations in the Tp-equivariant A'-group). Therefore, we do not prove the duality of 9. 
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We want to compute the ii"-group class of the DT complex (24) restricted to C 

E"^\c^Kl{C). 

A torsion free sheaf [E] G Cd has a T-equivariant structure as explained in Section]^ 
The complex restricted [E] is the virtual tangent space Tjj-j at [E] 

T[r] = E^i^{E,E) - E^i^{E,E) 


Here we used Hom(J^, J^)o = 0 and Ext^(J^, = Hom(J^, ® Kx) = 0, because E is 
stable and ^ 0. Also note that for toric varieties h^^iOx) = 0 so tracelessness 

is automatic. We write this as 


where we keep the trace-free part in the last term or else we would get a unwanted factor 
of Hom(7^, E) = C. The term (7^, E) — {E, E) is well-behaved and can be described by 
a nice vertex/edge formalism. The term —{E,E)o only contributes an overall constant 
in 

Let {Ua}a£V{x) be the T-invariant open affine cover of X and denote the double 
intersections by Uai 3 = UaP\Up. Denote the restriction of any sheaf E to Ua-, Uap by Ea, 
Ea^ respectively. Denote the global section functor by r(-). Using the local-to-global 
spectral sequence for Ext groups and computing sheaf cohomology by a Cech calculation 
gives 

— ^^( — 1 )* (T{Ual3, ^Xt\Eal3, Eafs)) — ^{UajS, ^xf{EE a/?)) j , 

where contributions of triple and higher intersections vanish because E and E only differ 
on a T-invariant closed subscheme of dimension < 1. Our goal is to calculate 

{Ea,Ea) - {Ea,Ea) = ^(- 1 )* ^T(Uq,, (7?.«, ) - T{Ua,S’xf{Ea,Ea))^, 

i 

{Eap,Eap) — {EaP,Eap) = ^ ( — 1 )* ^T (f/a/3, (7 ?.q,/3, ) — T {Uap, (S’xf {EaP, Eap))J ■ 


5.1. Vertex calculation. This section calculates 

{Ea,Ea) - {Ea,Ea)- 

Suppose the action of T on Ua = is in standard form {ti,t 2 ,t 3 ) ' = ti^i for all 

i = 1,2,3. Denote the Poincare polynomial of Ea by 

P{Ea)eK^{Ua)=Z[tf,tt,tf]- 

Consider the trace map 

tr : K^lUa) —^ Z((ti,t 2 , 0 )), 

which associates to a T-equivariant vector bundle its character (i.e. decomposition into 
weight spaces). Following the reasoning in |PT21 Sect. 4.4], we obtain 

P{Ea)P{Ea) 
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4.4 


where the operation (■) was introduced in Dehnition 

We need a family version of the above. This is easily done replacing by F, 7?. by 
p*x7l, Ua by Ua X C, and Uap by Uap x C. This gives 


The Poincare polynomial of p^^TZa can be expressed in terms of the equivariant param¬ 
eters ti as follows. Let (u, v,p) be the toric data associated to TZa (Definition |2.2[). 


Case 1: TZ^ is not locally free, or equivalently all Uj > 0 and all Pi are mutually distinct 
(Proposition |2.4[ ). Then 

P(r>* 'V \ ,U 2 +V 2 ^U2.+V3, I ^U^+Vl,U2^U2+V3, , ,U2+V2 _ MU-^+V-i ,U2+V2 ^U^+Vz 

^yPUa'^a) — ‘-1 >'2 ^3 “T ''2 ''3 “T ^2 ^3 ^2 ^3 

Case 2; TZa is locally free. In this case TZa has two homogeneous generators and 
PipljJTZa) is the sum of their characters. 


The character of Qq, is more complicated and involves moduli. Recall that C = (P^)^ 
for some > 0. Each factor P^ of C has a tautological subbundle C (Ppi © (Ppi 

attached to it. The character Qq, of Qq can be expressed in terms of the equivariant 
parameters ti and the classes of these C)pi(l)’s. Let tt correspond to Xa by Proposition 
[2T^ Then 

Qa ^ ^ Qa,(fcl,fc2,fc3)5 

(fci ,fc2 As) Stti U7r2U7r3 

where 

• Qa,(fciA 2 A 3 ) = when (fci, ^ 2 , ks) G tti fl 7r2 fl tts, 

• Qa,(fciA 2 A 3 ) ~ when (fci, ^ 2 , k^) lies in exactly one vr*, 

• Qa,{kiMM) ~ when (fci, A; 2 , k^) lies in exactly two tt^s but not in a region 

with moduli (see Section [2©] ), 

• Qa,{ki,k 2 ,k 3 ) ~ [C^c(l)]^i^^ 2 W when (fci,/c 2 ,^ 3 ) hes in exactly two vTj’s and in a 
region C = P^ with moduli (see Section 2.5), 

• Qa,(fciA 2 A 3 ) = 0 otherwise. 

Reasoning analogous to jFm Sect. 4.4] we obtain 
(34) 


which lies in 


QaPiPUa'k^o 

tlt2t‘i 


+ QaQa^— 


D)(l — ^2)(1 — H) 


tlt2t3 


TZo{C) ©z Z((^i, ^2, ts))- 


5.2. Edge calculation. This section calculates {TZap^TZap) — (Pa/s^Pais)- Assume the 
coordinates are chosen such that the toric line Ca/s H Ua is given by {x 2 = Xa = 0}. We 
work over the ring r(17Q^) = C[xf^,X 2 ,X 3 ] and define the formal ^-function 






Let 77 correspond to Xa by Proposition |2.18[ Denote the limiting double square config¬ 
uration along the xi-axis arising from tt by A = (Ai,A 2 ,A 3 ). Denoting the character of 
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Qa /3 by Qa/ 3 , we obtain 


where 


Qa/3 ^ ^ Qal3,{k2,k3)j 

(fc 2 ,fc 3 )SAlUA 2 UA 3 


• Qal3,{k2,k3) ~ h {k2, ^s) G Ai fl A 2 H A 3 , 

• Qa/ 3 ,(fc 2 ,fc 3 ) = if (^ 2 , ^ 3 ) lies in exactly one Aj, 

• Qa/3,(fc2,fc3) = f2^f3^ if (/c 2 , ^ 3 ) lies in exactly two Aj’s but not in a region with 
moduli, 

• Qap,{k 2 ,k 3 ) ~ if (^ 2 , ^ 3 ) hes in exactly two Aj’s and in a region C = 

with moduli, 

• Q«/3,(fc2,fc3) = 0 otherwise. 

Recall from Example |2.22| that from the data of A alone, we cannot determine whether 
a square lying in exactly two Ads has moduli or not. This depends on what happens “in 
the corners”. This already indicates we cannot split the vertex and edge contributions 
as easily as in the rank 1 case. We come back to this in the examples of the next section. 
A reasoning similar to the previous section gives 


(35) 


:= -Qa/3P(7^„^) - 


hts 




where PiTZa/s) denotes the Poincare polynomial of 7lai3- 


5.3. Redistribution. Exactly as in |MNOPlj and |PT2j . the terms of the previous two 
sections can be redistributed so they become Laurent polynomials in ti,t 2 ,t 3 . Denote 
the vertices neighbouring a G V{X) by / 3 i,/? 2 ,/d 3 - Define 

\j _I ,tiii) 

(36) 




Ea/3 . 


i ^ayih, ts) 


^al3{t2tl 


i=l 


ti 




i-tz^ i-tr 

where {^ 1 ,^ 2 , ^ 3 } = {ti, t*', V}. Here the labelling is such that Ca^- fl [/„ is given by 
{xi^ = Xi'i = 0} for all G {1,2,3} mutually distinct. Recall the definition of may, 

This redistribution does not lead to over-counting due to the 




from Section 


2.1 


choice of the second term in Eay (see ([^). The reason these are Laurent polynomials is 
the same as |PT2| Lem. 4]. We summarize this section in one theorem. 


Theorem 5.2. Let X be a smooth projective toric 3-fold with 7 ^ 0 and polar¬ 

ization H. Let A4x ■= -Ad;^(2, ci, C2, C3). Suppose C C Aix ® connected component of 
the fixed locus such that all its closed points have reflexive hull IZ. Then the T-character 
of DT theory % := E‘'^\c G Kq{C) is given by 

(37) trrj. = tr -( 71 , 71)0 + 

a£V(X) ay&E(X) 

where tr_( 7 ^^ 7 ^)(, G and for all a,fl E V(X) 

V„ G Kq(C) 

Ea/3 G Kq(C) 
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6. Applications to toric Calabi-Yau 3-folds 

Let Y be a smooth toric Calabi-Yau 3-fold and let A be a toric compactification with 
7 ^ 0 and polarization H. We £x ci, C 2 and consider the moduli spaces 


(38) 


Mx ■=\_\Mx(2,Ci,C2,Cz). 


C3 


In Section [4tT| we introduced the T-invariant open subset 

■MvcX ■= I_ICl) C2, C3) 


C3 


of torsion free sheaves A with T** jT supported in Y. We £x a connected component 
C C Mycx satisfying Assumption |4.7[ 


All closed points of C have the same (i.e. isomorphic) reflexive hull IZ. 


We introduced invariants DT(C) in Proposition-Dehnition 4.11 We are interested in 


the “Calabi-Yau specialization” si -|- S 2 -l- S 3 = 0 of these invariants. In this section, we 
conjecture a formula for generating functions of these invariants in terms of ^{q) 

of Remark 12.251 

Suppose [A] G C is a closed point with characteristic function corresponding to double 
box configurations tt (Proposition 2.18), where 

TTq, G n(77.Q, , ), 

for each a G V{X) with neighbouring vertices fli,fl 2 ,fl 3 - Since the cokernel Q = 1ZIF 
lies entirely in Y, the asymptotic double square conhgurations Xap are zero unless afl G 
EflY) (i.e. Cai 3 C Y) and = 0 unless a G V(Y). Note that A determines the second 
Chern class and 77 determines the third Chern class (Section 2.3, Proposition 2.24). 


Sl+.S2+S3=0 




Main Conjecture 6.1. Let Y be a smooth toric Calabi-Yau 3-fold with toric compacti¬ 
fication X satisfying 7 ^ 0. Fix a polarization H on X and Chern classes ci, C 2 . 

Let IZ be a T-equivariant rank 2 pi-stable reflexive sheaf on X with toric data (u,v,p). 
Suppose any connected component C of Ue3 (2, Cl, C 2 , Cs)^ has constant reflexive 

hulls IZ. For each afl G EflY), there are two faces pi^ajs, P 2 ,o /3 which share afl as an 
edge and two disjoint faces ps^a/s, ppap connected by the edge afl. Then 

E E DT(C) 

C3 CCM§^xi'^ri,C2,C3Y 

^_ 'Y'^X<y-l3\C^oiPirPl,al3~^'^P2,aP~^L+'^P3^aP~^'^P4,,aflq\^ap\^ap(ci{X) + C3) 

X aP&EpY) 

where C runs over all connected components and A = {A^/? G A(7Za/3)}ai3£EciY) runs over 
all double square configurations giving rise to second Chern class 02 - 

The assumptions of this conjecture are satisfied in numerous interesting cases e.g. Ex¬ 
amples |4^ and 4.10 We provide the following evidence for Main Conjecture |6.1[ 

• The examples of Sections 6.1 | 6.2 


For connected components C, “with expected obstructions” we compute the in¬ 
variants DT(C)|sj+s 2 +s 3 =o using methods of [HNOPI] (Section [ 6 .3[ ). The 
are compatible with Main Conjecture 6.1 


answers 
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• Assuming two analogs of conjectures in the stable pairs case [FTa Conj. 2, 
Conj. 3], we prove Main Conjecture 6.1 using To-localization (Section 6.4). 

Remark 6.2. Note that the formula of Conjecture 6.1| only depends on the compactih- 
cation through whether TZ is p-stable or not and through an overall multiplicative factor 
of q ". In particular, the formula 

A aqeE,{Y) 

does not depend on the compactihcation. We expect this formula to be of the form 
j\^(g,) 2 e(y) rational function in g, which is invariant under q O q~^ (up to an 

overall multiplicative factor). We prove this expectation in the case all A = 0 (no legs), 
where the rational function is in fact a Laurent polynomial (Remark |6.11| ). 

6.1. Rank 2 equivariant vertex without legs. In this section, we consider R = 
and IZ an arbitrary T-equivariant rank 2 reflexive sheaf on described by toric data 
(u, V, p). We will see below that TZ is always the restriction of a T-equivariant rank 2 /i- 
stable reflexive sheaf on a (polarized) toric compactihcation X of with ^ 0 

(Lemma 6 . 8 ). Nevertheless, we want to think of the results of this section as being 
intrinsic to C^. Since Tc(C^) = 0 , we choose all legs empty. Let tt G II(7^, 0 , 0 , 0 ). 
Then is the moduli space of T-equivariant rank 2 torsion free sheaves on with 
characteristic function corresponding to tt (Proposition 2.18). Even though we did not 
start with a compact 3-fold, the vertex expression (34) still makes sense for and is 
denoted by V^. 

Example 6.3. Suppose v = (1,1,1) and Pi,P 2 ,P 3 are mutually distinct. Consider tt 
given by 

Qtt = tits + ^2^3 + ti^ts -|- t2^t3 + tit2ts + ti^ts + + ^2^^1^3, 

SO 1 77 1 =8 (see Figure 6 ). Then we have 


t ^ 

Vtt =ts -|- tits + t2ts + ——-—— 


1 

tlt2 


i 2 

1 W 2 , -1 


-|- 3 ti -|- 3 ^2 


t2ts 

+ tr^-t 2 ""- 


tlts 

t2 


+ 


t2ts 

tits 


1 tp i2 


tits t2 


tl ^2^3 

t2ts tl 




t2^ 


+ 


tl 


+ 


tl% 


u 




tits t2^ts t2 t2^ts ti^t'- 


+ ^- 2 ^ 

ts ti^t2^ 

1 1 


to Lo. Lq 

+ 2-4+ - + 2-4 + - + - 


u 


+ 2 +^-«r‘‘- 

ts 

ts ts 1 


t2^ti 


t 2 '^ts tl ts 
1 tl^ 


tl 

1 


tl^t2ts^ 


t2ts 

- - 2 
t2 


t2%^ 

tl 


tlHs^ 

t2 


ts t2 tl t2 tit2ts 

1 ti^ ti^ t2^ t2 ^ I 


tit2^ts^ 

1 ^2 

tlt2^ ts 


t2^ts 


tlHs tiHs" 


and hence 


e(V, 


= 1 = 02 ( 77 ) 


Sl-|-S2-|-S3=0 


where 02(77) is the weight dehned in Dehnition 2.23 
Next we consider an example with moduli. 
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Figure 6 . Double box configuration of Examples 6.3 and 6.4 


Example 6.4. Suppose v = (1,1,1) and Pi,P 2 ,P 3 are mntnally distinct. We consider 
an example with | 7 r| = 4 and = P^, namely 

Qtt = tit2 + tits + ^2^3 + [0]pl{l)]tit2t3. 

See Fignre 6 . Then we have 

= 2[Opi(l)] - 1 - 2 + ■ ■ ■ , 

^ ^ tlt2t3 tit2t3 

where ■ ■ ■ indicates the To-moving terms. Since Tc^ = 2[C>pi(l)] — 1, we obtain 


V„ = 2 


[Opi(-l)] 


+ 


^ 1 ^ 2^3 tit2t3 

A direct calcnlation shows that ■ ■ ■ prodnces a sign —1. We conclnde 


e(Tc„-V„)| 


Sl+S2+S3=0 


= 2 = uj(tt). 


These two examples snggest 


^Ctt 

for any tt G n(7^, 0 , 0 , 0 ). However this is false as is shown by the following example. 

Example 6.5. Snppose v = (1,1,1) and pi,p 2 ,p 3 are mntnally distinct. We consider tt 
and tt' given by 

Qtt = tits -|- ^2^3 + -|- t2^t3 + ^^2^3 + ^^2 + ^^^2^3 + t2^tit3^ 

Qtt' = tits + t2ts + ti^ts + t2^ts -|- ^^2^3 + ti^t2^ts + ti^t2ts + t2^tits^ 

SO |7r| = Itt'I = 8 (see Fignre 7). Note that neither C.„ nor has modnli. We obtain 


Vtt' — tit2ts 


1 


J.2J.2J.2 

^l''2''3 


+ 


where ■ ■ ■ indicates the To-moving terms. Fnrther calcnlation shows 

1 


e(V^) 

1 


Sl+S2+S3=0 


e(V ^0 


Sl+S2+S3=0 


= 0 ^ a;(7r) = 1, 
= 2 ^ uj{7t') = 1. 


However, we do have 


1 1 
+ 


e(V^) e(V^0 


Sl+S2+S3=0 


= 2 = 02(77) -|- 02(77') 





























RANK 2 SHEAVES ON TORIC 3-FOLDS 


51 




Figure 7. Double box configurations of Example |6.5 


In |GKY] . we associate to any double box configuration tt a double dimer model D(7r). 
It turns out that in this example 

D(7r) = D(7r^). 

In analogy to |MNOPll IPT2] . we define the rank 2 equivariant vertex measure: 

Definition 6.6. Let 77 be a T-equivariant rank 2 reflexive sheaf on C^. For each tt G 
n(77, 0,0,0) define the rank 2 equivariant vertex measure associated to tt by 

w(7r) ;= f e{Tc^ - V^). 

Define the rank 2 equivariant vertex as 

(39) 0,0, 0 (g) := ^ w(7r)gl’"l e Z(si, 52 , 53 )[g]. 

7 ren{ 7 ^,0,0,0) 


It is not a priori clear the “Calabi-Yau specialization” 5i + 52 + 53 = 0 of the rank 2 
equivariant vertex is well-defined. 


Proposition 6.7. Let TZ be a T-equivariant rank 2 reflexive sheaf on C^. Then 


is well-defined. 


W7^,0,0,0(g) 


eZW 

Sl-|-S2-|-S3=0 


Proof. By Lemma |6.8| below, this is a special case of Proposition-Definition |4.11 
only have to note that each C„ satisfies Assumption 4.7 


We 

□ 


Lemma 6.8. Let TZ be a T-equivariant rank 2 reflexive sheaf on Y = C^. We can view 
Y as a T-invariant affine open subset of X = (P^)^. There exists a polarization H on 
X and a T-equivariant rank 2 p,-stable reflexive sheaf IZ on X such that 77|y = 77 and 
77 restricted to any other T-invariant affine open subset is locally free. 


Proof. Let (u, v, p) be the toric data of 77 and denote the pull-back of the class of the 
point via projection to each of the three factors of X = (P^)^ by Di, D 2 , D^. We prove 
the case 77 is singular, i.e. vi,V 2 ,v^ > 0 and pi,P 2 ,P 3 ci-re mutually distinct (Proposition 


2.4). The degenerate cases can be treated in a similar fashion. Let 

H = aiDi Y 021)2 -|- a^D^ 


be any polarization on X, i.e. 01,02,03 G Z>o. Define 77 by the following toric data. 
For each of the faces p G A(X) corresponding to the chart Y = C^, we take Up, Vp, and 
Pp equal to the corresponding value of Ui, Vi, and pi coming from 77. For each other 
p G A(X), we take Up = Vp = 0. Then clearly 

n\Y = TZ 

and iZ restricted to the other standard charts is locall y fre e (Proposition |2.4[ ) . It suffices 
to ensure TZ is /i-stable. This is equivalent to (Section |2.3[ ) 
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for all i,j, k G {1, 2, 3} mutually distinct. These inequalities are equivalent to 

2ajakVi < 2aiakVj + 2aiajVk, 

for all i,j,k G {1,2,3} mutually distinct. Taking ai = Vi, 02 = V 2 , 03 = U 3 proves the 
lemma. □ 

Conjecture 6.9. Let 71 be a T-equivariant rank 2 reflexive sheaf on C^. Then 


^'R,0,0,0(,q) 


Sl+S2+S3=0 


~ Zc3,'R.,0,0,0(?)- 


Remark 6.10. Recall from Section 2.6 that 0 , 0 , 0 (g) is just the generating function 
of Quot schemes 


^e(Quotc3(7^,n))g^ 


n=0 


There are two essentially different cases of Conjecture 6.9 


(1) 71 is locally free, i.e. some Uj = 0 or some pi are equal (Proposition \2A\ . Then 
Conjecture 6.9 and equation (15) imply 


W7e^0_0^0(g) 


Sl+.S2+S3=0 


= M{qf 


(2) 7Z is singular, i.e. all Uj > 0 and Pi are mutually distinct. In this case (39) only 
depends on v (by (34)) and we dehne 


W, 


Vl,V2,V3 


, 0 , 0 , 0 ( 0 ') •— ^TZ,0,0,0{(l)- 


The right hand side of Conjecture 6.9| is computed in Theorem 2.27 Conjecture 
ig no] 

Vi^V2,V^,0^0,0 (?) 


6.9| is an interesting non-trivial combinatorial identity stating 

W, 


^ 1 _ qi+j+k-i 


Sl+S2+S3=0 


A/(#nnnTv 


6.9 


Remark 6.11. Conjecture 
legs). In this case, the LHS of 


i=l j=l k=l 

implies Main Conjecture 


1 -q 


i-l-jfl-k—2 ' 


Vlain Conjecture 


6.1 


equa 


6.1 


s Z 


in the case all A = 0 (no 
y, 7 ^|y(g“^) up to an overall 


multiplicative factor of q to some power. Since there are no legs, the formula of the 
previous remark shows the generating function is times a Laurent polynomial 

in As can be checked by a small calculation, this Laurent polynomial is invariant 
under q GA q~^ (up to an overall multiplicative factor). 


Remark 6.12. In Example 6.5, we saw that in general w(7r) 7 ^ 00 ( 77 ). However, we do 
expect the following rehnement of Conjecture 6.9 to be true. Let 77 be a T-equivariant 
rank 2 reflexive sheaf on C^. In |GKYj . we associate a double dimer model D(7r) to any 
77 G n(77, 0, 0, 0 ). For any double dimer model D, we conjecture 


E 


w(7r) 


77 e ^(7^, 0,0,0) 
D(7r) = D 


Sl+S2+S3=0 


= 


E 


a; 77 


)?' 


ttI 


77 e n(7^, 0,0,0) 
0 ( 77 ) = D 


In [GKYj . we refer to elements n( 77 , 0,0, 0) with the same double dimer model as 
buddies. In Section 6.4, we give a (conjectural) geometric characterization of buddies: 
771,772 are buddies if and only if T/To-hxed connected components inside 

the same To-hxed connected component (Remark 6.20[). 
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6.2. Rank 2 equivariant vertex/edge for one leg. For V = C* X C^, one can fix 
a double square configuration A along C* (Definition 2.14). Unlike the rank 1 case, the 
edge term (36) along this leg is not determined by A alone. Therefore, Ea is not well- 


dehned. The reason is that whether the leg has moduli or not depends on what happens 


in the “corners attached to the leg” (Example 2.22). In this section, we consider 


U = Tot(C)pi(-l) ©C)pi(-l)). 

Let 7Z he a T-equivariant rank 2 reflexive sheaf on V described by toric data (u, v,p). 
Let 0,1 label the two vertices of A(U). The polyhedron A(U) has four non-compact 
faces. Two of them share the unique compact edge and we denote them by pi, P 2 - The 
other two we denote by P 3 ,Pi, where ps has vertex 0 and p^ has vertex 1. We will see 
that TZ is always the restriction of a T-equivariant rank 2 /i-stable reflexive sheaf on a 


(polarized) toric compactification X oi Y (Lemma 6.15). In order for this extension of 
TZ to have no T-flxed moduli, we assume 


= 0 . 


Fix a double square configuration A along the compact leg and tt = {7rQ,}a=o,i) where 

7r„ G ^(7^„, A, 0,0). 

Then tt has at most 3 face components and can have several leg and vertex components 
of I’s (Definition |2.7[ ). Therefore, there are no moduli associated to the face components. 
In particular, the moduli space (Definition |2.23| ) decomposes according to vertex and 
edge components 

^TT — f^vert ^ Cedge- 


Note that the expression for the edge in (35), (36) makes sense without reference to a 
compactification of X. We write the corresponding expression as E.^.. It is tempting to 
define an equivariant edge measure 


(40) 


e{Tc. 


edge 


E*). 


-'edge 


However, this may not be well-defined as illustrated by the following example. 

Example 6.13. Suppose 7Z is given by toric data (u,v,p) with (up^,Upj,Upg,Up^) = 
(1,1,1,0) and pi,P 2 ,P 3 mutually distinct. We consider the following three double box 
configurations tt, tt', tt" with leg along the zero section of Y 


(Qtto) Qtti) — ( tit 2 + 


+ ^2 + 2tit2) tit^ -|- ^2^3 T 2tit2t 


I.-2 


1-^3^ 


(Qtt© Qtt') — ^^1^2 + 


1 — ^3 

tsitl + ^2 + [C^pi (1)]U^2) U^3 ^ + ^2^3 ^ + [C^pl (1)]U^2^3 ^ 


(Qtt", Qtt") = 


1-^3 

4-1 I 4 4-1 


1-t 


-1 


tsitl + ^2 + U^2) + ^2^3 + tit2^3 


-1 


l-h 


1 - t 


-1 


The double box configurations in chart Uq are depicted in Figure 8. One can compute 


/ 1 1 1 1 1 1 ti ^2 „ \ 

^ tf ti ^2 U^3 0^3 ^2^3 tlH' 


n, ^ , 1,1 [(^P^(l)] 1 , 1 [(^Pl(l)] 


2 ^ [Op. (1)1 , 2 

UO U^2 ^3 
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Figure 8 . Double box configurations 7ro,7rQ,7rg of Example 6.13 


[C>pi(-1)] 


1 I IC’r.(-l)] 


^ 2^3 ^ 2^3 ^ 2^3 ^ 1^3 


(V^., = + — 

^ ° V ti h hts t2h hhh 

_2fi 2^2 _ 

12 


,0 


^1^3 ''2''3 




E =2[C*jt.i(l)] 1 I I [C>pi(-1)] [C>pi(-1)] 


to 


tH, 


Ds 


tU. 


2''3 


2[Opi(-l)] ^ 1 


^1^2^3 

E*,.=l+^ + ^- 


^1^2^3 ’ 
1 1 


^2 ' ^1 ^?^3 t'its tit2h 


Interestingly, E^// has a positive T-fixed part, so l/e(E^//) is not well-defined. Therefore 
(40) is in general not well-defined. As a consistency check: an explicit calculation using 


these expressions leads to the sign predicted by Main Conjecture 6.1 


In the previous example, the obstruction term in cancels after adding We 

should therefore not separate these three terms. This motivates the following definition. 

Proposition-Definition 6.14. Let Y = Tot((!lpi(—1) ©(!lpi(—1)) and label the vertices 
and faces of A(F) as before. Let 71 be a T-equivariant rank 2 reflexive sheaf on Y 
described by toric data (u, v,p) with Vp^ = 0. Let X be a double square configuration 
along the compact edge and let tt = {7rQ,}Q=o,i; where 

TTa G n(7^Q,, A, 0, 0), for a = 0,1. 

Then 

w(7t) := [ e{Tc^ - (V^o -h E^r)) G Q(si, S 2 , S 3 ) 

JCi, 

is the rank 2 equivariant resolved conifold measure associated to tt. The specialization 

w(7r) G Q 

Sl+S2+S3=0 

is well-defined. Define the generating function of these measures as 

y^Ti,\,0,0iq) = e Z(si,S2,S3)M, 

TTen{TZ,x,0,0') 

where /_i _i(A), Qup ,up ,vp ,o(A) were defined in Definition 


2.23 
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Proof. We have to prove the Calabi-Yau specialization si + S 2 + S 3 = 0 is well-dehned. 
The proof is similar to that of Proposition 6 .71 and follows from the following lemma and 
Proposition-Dehnition 4.11 □ 


Lemma 6.15. Let IZ he any T-equivariant rank 2 reflexive sheaf onY = Tot((Ppi (—1) © 
(Ppi(—1)) described by toric data (u, v,p) (not necessarily with Vp^ = Oj. Then there 
exists a polarized toric compactification X of Y and a rank 2 n-stable reflexive sheaf IZ 
on X, such that TZ\y — TZ and the restriction oflZ to any T-invariant affine open subset 
not contained in Y is locally free. 


Proof. We treat the case all Vi > 0 and all Pi are mutually distinct. The degenerate cases 
can be done similarly. 

Let Fi = P(C>pi © C>pi(l)) be the hrst Hirzebruch surface and let C be the T-£xed line 
with = —1. Let F be a T-hxed hbre. Define 


TT : Y = P((Pf, © C>P,(F)) Fi. 

The following divisors form a Z-basis for Pic(X) 

Di = 7r-^(C'), D 2 = n-\F), D 3 = P((Pfi © 0) C X. 

Then C is a (-1,-1) curve in X contained in two toric charts Uo,Ui. Their union is 
isomorphic to Y. We dehne 77 on X by the following toric data (u, v,p). The faces 
P11P2-, p3, Pa of A(X) (introduced at the beginning of this section) correspond to faces of 
A(X) and we take Up^,Vp^,Pp. from the given toric data (u, v,p). For all other faces p 
of A(X), we set Up = Vp = Q. Then 77|y = 77 and 77 restricted to any other other toric 
chart is locally free (Proposition |2.4[ ). We are reduced to finding a polarization 77 on X 
such that 77 is /i-stable. 

The intersection numbers among the toric divisors are 

D 1 D 2 D 3 = pt, DiDI = = —pt, 

Dl = DID 2 = DID 2 = Dl = Dl = 0, 
where pt denotes the class of a point. A divisor 

77 = aiDi + (oi + 02 + < 23)772 + a^D^ 

is ample if and only if oi, 02 , 03 > 0. Take A G Q>o arbitrary and Oi = Aui, 02 = ^2 + ^ 4 , 
<23 = V 3 . Using the above intersection numbers, it is not hard to see that there exists a 
^ Q>o such that the following four stability inequalities hold 

{H‘^Di)vi < {H'^Dflvj + {H‘^Dk)vk + \/i,j,k,l e {1,2, 3,4} mutually distinct. 

After scaling 77 such that all its coefficients are integer, we are done. □ 


Parallel to Main Conjecture |6.1 we conjecture: 


Conjecture 6.16. Let Y = Tot((!7pi(—1) © (!7pi(—1)) and label the vertices and faces 
of A(Y) as before. Let TZ be any T-equivariant rank 2 reflexive sheaf on Y described 
by toric data (u, v,p) with Vp^ = 0. Let X be a double square configuration along the 
compact edge. Then 


W 


'R.,\,0,0 


( 7 ) 


Sl-I-S2-I-S3=0 


= (^_X)I'^IKi+'"P2+'"P3+1)Z 


'Jl,X,0,0 


(7). 
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6.3. Expected obstructions. In this section, we explain where the sign in Main Con¬ 
jecture 6.1| conies from. The following theorem discusses the contribution of connected 
components C with “expected obstructions”. The argument is a variation on |MNOPll 
Sect. 4.10, 4.11]. 

Theorem 6.17. Let Y be a smooth toric Calahi-Yau 3-fold with toric compactification 
X with 7 ^ 0 and polarization H. Let TZ be a T-equivariant rank 2 p,-stable 

reflexive sheaf on X with toric data (u, v,p). For each afl G EflY), there are two faces 
Pi,a/ 3 ? P 2 ,ai 3 which share afl as an edge and two disjoint faces pz,ais, p 4 ,ai 3 connected by 
the edge afl. Let fr G n(77|y,A) for some X = {Xa/s G X(7lai3)}ai3£EpY) and consider 
the connected component C = Cir- Assume the following: 

(i) All closed points of C have reflexive hull IZ. 

(ii) The To-fixed part of Eaey(y) + T^ageE^Y) Ea/3 equals Tc - lie ® itit 2 h)-^. 
Then 

DT(C) = e(C) n I (va^p +-«P 2,«/3 +b+’'P 3 ,c /3 +'"P 4 ,c «/3 ) . 

ag&EpY) 

Remark 6.18. In the case C is in addition isolated, (ii) states that J2a&v{Y)'^a + 


Z]a/3eE,(y) 


has no To-£xed terms. In |MNOPl] . the T-hxed locus automatically 
consists of isolated reduced points and the absence of Tg-fixed terms always holds (this 
is the crucial technical point mentioned in |MNOPll Sect. 4.11]). In our case, C could 
be non-isolated. Moreover, even when C is isolated there can be non-zero Tg-hxed terms 


as for 77 and tt' in Example 6^ Examples 6^, 6T, and 6.13 satisfy the assumptions of 
the theorem. 


Proof of Theorem 6.13\ From the dehnition and assumptions 

DT(C) = [ e{klc ■ ■ ■ , 

JC Sl-|-S2+S3=0 

where ■ ■ ■ stands for the contribution of all Tg-moving terms. This is equal to 

DT(C)= [ ctop(f^c)--- 

Jc si+S2+S3=0 

Because of degree reasons, the contribution of ■ ■ ■ is simply ±1. More precisely, it is —1 
to the power the number of (Serre dual) Tg-moving pairs in Yha&viY) + J2ageEc{Y) 
by Proposition |4.6[ We conclude 

]I)T(C) _ (_ ]^^dim{C)+number of To-moving pairSg^^^ 

In order to determine the sign, we use the following claim. For all afl G EflY) and 
a G ViY), there exist splittings 

(41) c „ - c+ 

(42) 

such tha0 

(43) 

(44) 


v„ = vi + v;, 

Eq« “ -(^1^2^3) ^E 


'a/3; 


v„ = -itit2t3) 


^'^Note that [MNOPl] only show these dualities after specialization tit 2 t 3 = 1- Our dualities also 
hold before specialization. 
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If the claim is proved, then we conclude DT(C) equals 

where Constant(- ■ ■) denotes the rank of the To-hxed term. Moreover, V+(l, 1,1) and 
E^^(l,l,l) should be interpreted as hrst restricting the vertex and edge expression to 
(any) closed point of C, thus getting rid of moduli, and then putting {ti, ^ 2 ,^ 3 ) = (1,1, !)• 
So we need to prove the above splittings and compute the terms in the power modulo 2. 
For (41), we define (similar to |MNOPlt Sect. 4.10]) 


—QapPiJ^a^) ~ Qa/^Q 


^ l-to 


OLp 


In this dehnition, the coordinates are chosen such that fl Ua is given by X 2 = = 0. 

We then define as in expression (36), but with replaced by Also define 

E“^ := Eq ,/3 — E^^. The identity (43) can then be checked by direct calculation. This uses 
ma /3 + = —2 (valid because a(3 G Ec{Y) and Y is Calabi-Yau). From the definition 

of E)^^, we obtain 


Ei(l, 1,1) = - G+3(f3r’".»,f3r'“U)) 




= 


( 1 , 1 ) + (mapt2-^ +rn'^^t3-^^G+^{t2,h) 


d 


mod 2 


(ma0t2^ +m'^^h^^G+^{t2,t3) 


d 




(t2,t3) = (l,l) 

mod 2, 


where the second equality follows from P(Jla/ 3 ){l, 1,1) = 0 mod 2 {7Z has rank 2). Using 
the fact that P(7 ^q,/3 )(1, 1,1) = 0 mod 2 and = —2, one obtains 


(45) 


Ei(l. 1.1) = ’"ofll-'ouKfp,,., + + 1) mod 2, 


where Pi,q/ 3 , P 2 ,a /3 ^ E{X) are the two faces with common edge al3. 

The splitting (42) is more complicated. Let G^^ be dehned as above. Naively, one 
would like to define 


V+ ;= Q„P(P„) - Q„Q 


(l ti)(l-t2) 








i=l 


and V“ := Vq, — V+. Here the vertices neighbouring a G V{Y) are denoted by (3i, 132, (3^, 
where the labelling is such that Uq/j. fl Ua is given by {xi> = Xi>/ = 0} for all i,i',%” G 
{1,2,3} mutually distinct. In the case |Aq,/ 3 | = 0 (no legs), this works and one obtains 
(42). The general case is more complicated. One introduces Q'^, by the equation 


Q= = QL + Etv 


2=1 


ti 


and defines V+ exactly analogous to |MNOPlt Sect. 4.10]. Setting V^, := Vq — V+, 
equation (42) can be deduced. The correct dehnition of V+ leads to 


V+(l, 1,1) = Q'„(l, 1,1)P(P„)(1,1, l)+5^ ^(^«ft)) 


2 = 1 




mod 2. 
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This equality follows after noting that various terms in V+ are zero or even for ts) = 

(1,1,1). A similar (but easier) calculation as for shows 

3 

(46) V+(l, 1,1) = ^ \K 0 ^Vp^ mod 2, 

^=1 

where pi, p 2 , Ps are the three faces with vertex a labelled such that Pi'^Pi" are the faces 
sharing edge ajSi. 

Finally, we claim 

Constant(V^) + Constant(E^^) = dim(C) mod 2. 

aeV(Y) al3£EPY) 


'^al3- 


Dehne 

E E 

aeV{Y) al3eEciY) 

Take any To-hxed term / in not occurring in Tc — hlc ® (^i^ 2 ^ 3 )~^- This term must 
cancel after adding X” (by assumption). Therefore, X^ contains the term —/. By 
duality (43) and (44), X+ must contain f*{tit 2 t^)~^. Therefore, nonzero terms in X+ not 
occurring in Tq — ® (^i^ 2 ^ 3 )~^ come in pairs (and likewise for X^). Distributing the 

terms of Tc — Vtc® (tit 2 ^ 3 )~^ over X^ by using (43), (44) and calculating modulo 2 shows 
the claim. □ 


6.4. To-localization. In this section, we prove Main Conjecture [6]^ from a To-localization 
argument similar to |PT2l Sect. 5.2]. Pandharipande-Thomas need two conjectures for 
their argument |PT21 Conj. 2, Conj. 3]. We need completely similar analogs of their con¬ 


jectures in our setting. The hrst is smoothness of the To-fixed locus (Conjecture 4.12). 
The second will be described below. 


Theorem 6.19. Assume Conjecture 4.12 is true. Let Y be a smooth quasi-projective 
toric Calabi-Yau 3-fold with toric compactification X satisfying ^ 0. Fix a 

polarization H on X and let Aix ■= A 4 xi‘^XiX 2 ,C 3 ) and AAycx '■= AA.ycx {‘^^ D; C 2 ; 03 )- 
Suppose C C Xi'^ is a connected component satisfying Assumption 4-13 with reflexive 
hull IZ. Let be the virtual normal bundle on C w.r.t. the To-action. Then the class 

+ (7^,7^)o®(Pc e K^flC) 

is of the form N~^ N~ with N = —N~^ G Kq°{C). Moreover 

DT(C) = (-l)^™(^)+’’’^(^^)e(C). 


Proof. By symmetry of the To-equivariant obstruction theory (Theorem 4.17) 

= Tc - Dc + 

Therefore, the invariant is given by 

DT(C) = j^e(Dc - + (77,77)o ® Oc)). 


Note that (77,77)o only has To-moving terms by Assumption 4.13 Theorem 4.17 therefore 
gives 


Ar™’0 + (77,77)o = -(ATvinO + (77,77)o ® Oc) e K^flC). 
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Consequently, + {TZ, 1Z)q is of the form iV+ + N with N = —iV+ and we conclude 

DT(C) = J □ 

Remark 6.20. Combining Theorem |6. 19 and Proposition |4.16| gives 
DT(C) = (^_X)'iim(C)+rk(A+) E ^(c) = E DT(Ci), 

CiCCC* CiCCC* 

It is tempting to conclude 

DT(Ci) = 

for any C*-£xed component C* of C. This is certainly not true in general, because it 
would imply DT(Ci) is never zero contradicting Example 6.5 This is very reminiscent 


where T/Tq = 

(47) 


of the discussion of buddies in Remark |6.12[ This leads us to expect that two elements 
TTi, 772 £ n(7?.. A) are buddies if and only if lie in the same connected component 

of the To-hxed locus. This would provide a geometric explanation for the (combinatorially 
mysterious) notion of buddies. So in general the classical count ±e(Cj) and virtual count 
DT(Cj) are different. Remarkably, they become equal again after summing over all 
connected components Ci of the same connected component of the To-hxed locus. 

The analog of the second conjecture of Pandharipande-Thomas |PT21 Conj. 3] is: 


Conjecture 6.21. Let Y be a smooth quasi-projective toric Calabi-Yau 3-fold with 
toric compactification X satisfying ^ 0. Fix a polarization H, let Mx ■ = 

A4x(2, Cl, 02 , 03 ), and let Mycx ■= 01 , 02 , 03 ). Suppose C C AlJ is a con¬ 

nected component satisfying Assumption f.l3 and let Ci be a connected component ofC^* 
where C* = T/Tq. Since Ci is a connected component of AAx, we have defined vertices 
V., VJ and edges Eap, on it. For any [X] G Ci, these give restrictions Vci|[j-], V^|[j-p 
Ea/ 3 |[j-], E^^|[j-]. Then 

dimExt^(T', ^ Constant (V^ | )-|- ^ Constant (E+^|[j-]) mod 2, 

aeviY) ayeEciY) 

where Constant(- ■ ■) denotes the rank of the Tp-fixed term. 

Remark 6.22. We recall that the quantities and E^|[j-] were defined in the proof 

of Theorem 6.17 Their dehnition and equations (45), (46) hold under the assumptions 


of Conjecture 6.21 and do not need the additional assumption (ii) of Theorem 6.17 


Remark 6.23. Suppose the setting is as in Conjecture 6.21 for some Tp-fixed conn ected 
component C. If Cj is a conn ected component of C’^* s atisfy ing (ii) of Theorem 6.17, then 
it satishes Conjecture 6.21 by the proof of Theorem 6.17 If C = C'^* then C is smooth 


(nam ely a product of P^’s) so it satishes Conjecture 4.12 See Exam ples 6.3, 6.4, and 
6.13 If C 7 ^ , we also ha ve e vidence supporting Conjecture 4.12 e.g. the Tp-hxed 

terms of tt and tt' of Example |6.5| suggest that C = P^ and C^* consists of its two T-hxed 
points. 


Conjectures 4.12 and 6.21 imply (a slightly weaker version of) Main Conjecture 6.1 


Theorem 6.24. Let Y be a smooth quasi-projective toric Calabi-Yau 3-fold with toric 
compactification X satisfying Fl'^{Kf}) 7 ^ 0. Fix a polarization H on X and Chern 
classes ci, C 2 . Let TZ be a T-equivariant rank 2, fv-stable reflexive sheaf on X with toric 
data {u,Y,p). We assume: 
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•5l+S2+S3=0 


= 


(i) Conjectures j.l2 and 6.21 hold. 

(ii) = 0. 

(iii) All the connected components C C Uea PCX ( 2 , Cl, C 2 , 03 )'^° are compact and have 
constant reflexive hulls TZ. 

For each afl G EflY), there are two faces pi^a^, P 2 ,ai 3 which share a(3 as an edge and two 
disjoint faces ps^a/s, P 4 ,ai 3 connected by the edge afl. Then 

E E DT(C) 

C3 CCX!^^jf(2,Cl,C2,C3)^ 

^_]_y'^Q/3K™-a/3('ypi_(j^+'yp2,c«/3“*"^)“*"''^P3,a^“*"''^P4,a/9 )gl'^a/9|GQ,^(ci(X)+Ci) 

A ag&EpY) 

where C runs over all connected components and X = {Xap G XiflZap)}ag&Epy) runs over 
all double sguare configurations giving rise to second Chern class c^. 

Proof. The LHS can be written as 

E E DT(C)««, 

C3 CCX!|t^_y(2,Ci,C2,C3)^0 

by Proposition 4.16 and where the snm is over all connected components of the To-fixed 
locns. By Theorem 6.19| 


DT(C) = (-l)dim(C)+rk(A+)g('(^^ ^ ('_^^dim(C)+rk(A+) ^ 

CiCCC* 

where the snm is over all connected components of C'^*. Snch a component Cj = for 
some TT G n(7?.|y, A) for some A = {Aq,/? G XiJZais)}ag&Epv) giving rise to second Chern 
class C 2 . If we can show 

dim(C) + rk(Ar+|c^) = ^ -1- + 1) + Vp^^^^ + Vp^^^p) mod 2, 

ageEpY) 

then we obtain RHS. 

We evalnate dim(C) -I- rk(iV+) on an arbitrary closed point [IF] G C#. By dehnition of 

'v*. VJ, 

rk(/V+||p])= E (Vj|[p|(l,l,l) - Constant(V+||p])) 

aeV{Y) 


+ -Constant( 


-a/3 IT] 


-]))• 


ageEPY) 

In the proof of Theorem 6.17 we compnted 

E v:i[p,(i,i,i)+ ^ E+ji[p,(i,i.i) 

a£V(Y) ageEciY) 

= Yl I^«/3|("^«/3Ki,o/3+W2,./3 + 1)+V<./3+V./3) 

ag&EpY) 

The theorem follows from 

dim(C) = Constant(V^|[j-]) -f Constant(E^^|[jr]) mod 2, 

aeViY) ag&EPY) 

which is precisely Conjectnre 6.21 


□ 
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Remark 6.25. Let y be a smooth toric Calabi-Yau 3-fold with toric compactihcation 
X satisfying ^ 0. Let IZhe a. rank 2 /i-stable reflexive sheaf on X with respect 

to some polarization. Assume: 

(i) Conjectures |4.12 6.21 hold. 

(ii) Ext^(7^, 7^)^° = Ext (71,71)'^° = 0. Let C C a compact connected 

component with constant reflexive hulls 71 and all cokernels 0-dimensional. 

Then DT(C) = (—gy calculation in the proof of the previous 
theorem, (^_ijdim(c)-i-rk(A+) _ (no legs). We deduce that in the absence of legs 


DT(C) = DT(C) 

CiCCC* 


e{Q = e{C). 

This formula holds even though in general DT(Cj) 7 ^ e{Ci) (Example |6.5[). 


E 

CiCCC* 
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